UNIX for Dummies Questions & Answers

I would like to question the use of for-loops to cycle through arbitrary lists.

As long as a list is of fixed length there is nothing problematic with a for-loop:

Code:

for x in a b c d e ; do
     do_something $x
done

The problem arises when shell-regexps ("globs") are used to expand to a list of files:

Code:

for file in $(ls some_pattern*) ; do
     do_something $file
done

In this case there is a chance of the script failing: the glob is expanded to a list of files before executing the commandline (this is shell standard) and - depending on the numbers of files it expands to - the command line could get longer than the maximum supported command line on POSIX systems (which is 4k characters, see sys/limits.h). In this case you run a chance of the shell terminating the script with a "command line too long" (or similar) error. This is true for ksh, bash and sh alike, probably for most of the other shells too.

What can be done to avoid this? The solution is quite simple: replace the "for x in ..."-construction with an equivalent while-loop and feed that with a pipeline. The second example from above could safely be rewritten as:

Code:

ls some_pattern* | while read file ; do
     do_something $file
done

I hope this helps.

bakunin

Thanks!
I can't make it work though...
Where do I substitute with my real file name(s)?
I tried to combi the two codes and it didn't work. What is it I do wrong?

Code:

ls *contacts | while $file ; do awk '{printf ($2 in a) ? ","$5 : (NR>1) ? RS $2 FS $5 : $2 FS $5; a[$2]} END{print e}' $file done

The prolem is in the first line: sustitute "file" for "$file$ and add a "read", which is a keyword. See my example above. The first line schould be

"ls *contacts | while read file ; do"

and the variable "file" will hold one value after the other in each cycle of the loop. The following example is to make it better understandable:

Code:

ls *contacts | while read myvar ; do
     echo "===== $myvar ====="
done

Compare the output of this to the output of "ls -1 *contacts". You will notice that "myvar" gets filled with one value after the other in the loop as you can see when it is used in the echo-statement.

I hope this helps.

bakunin