Shell Programming and Scripting

It is a long standing knowledge that such an approach (even if the syntax errors are corrected, because it is NOT Do...Done but do...done - the language is case-sensitive) will always be way slower than using awk (or sed or any other text filter) on the whole file.

The reason is: whenever you call an external program (external to the shell, that is) from the shell you start a new (sub-)process. Starting a process is a resource-consuming activity for the system: it has to load an executable into memory, allocate the resources (memory, etc.) necessary to run it and finally start it. This:

Code:

command

is exactly one such process, while this:

Code:

while read line ; do
     command
done < /some/file

will create such a new process for every line in the input file. When you say the file is 70MB big i suppose these are alot of lines.

Of course, the opening of a single process is no big deal. It will add up, though, and "no big deal" times several thousand times eventually adds up to some big deal.

I hope this helps.

bakunin