Shell Programming and Scripting

Hi,

I have written a tcsh script that reads 3 directories, lists the files within them and continues on to evaluate each file independently. When I run the script, it gets to a certain directory that contains ~250 files, when you echo the directory, there are 4332 characters. So, when I use the echo command to list the files within that directory within the shell script, it produces a "word too long" error. The confusing thing is that when I run the commands directly in the shell (i.e., not through a script, type each command separately), I don't get the error. Is there anyone out there that can help me understand why the script fails, but running the commands directly in the shell does not result in an error?

Here is the list of the directories:
drwxr-sr-x 2 4096 Jan 21 10:37 SRS00001
drwxr-sr-x 2 4096 Jan 20 18:27 SRS00008
drwxr-sr-x 2 4096 Jan 21 11:04 SRS00009

Here is the section of the script (and output) that is failing:

set dlist = ();
set flist = ( `ls -d *` )
foreach f ($flist)
if ( -d $f ) set dlist = ( $dlist $f )
echo "dlist = $dlist"
------------------------------------------------------
output of the echo command for each iteration:
dlist = SRS00001
dlist = SRS00001 SRS00008
dlist = SRS00001 SRS00008 SRS00009
------------------------------------------------------
end

foreach d ( $dlist )
set flist = ( `find $d -name "*"` )
echo $flist
------------------------------------------------------
output of the echo command for each iteration:
SRS00001 SRS00001/IMG00000 SRS00001/IMG00001 ... SRS00001/IMG00049
SRS00008 SRS00008/IMG00000 SRS00008/IMG00001 ... SRS00008/IMG00024
SRS00009 SRS00009/IMG00000 SRS00009/IMG00001 ... SRS00009/IMG00247
------------------------------------------------------
end

As you can see, the last directory contains 248 images, resulting in an echo output of more than 4000 characters. I assume this is why I get the "word too long" error when I run the script, but why does it work when I manually type in the command in the shell?

I've been struggling with this for 2 days, and the script won't continue once it hits this directory, so I can't move on from this point. If anyone can help, it would be greatly appreciated. Thank you in advance!

nix

The length of the available length on a command line varies depending on server memory and how the OS allocates resources to a process. It might be smaller in a shell script because memory has already been used for other things.

The typical work-around include (1) using find, (2) using shell pipes instead of glob-expansions to iterate over a list of files, (3) break up the glob into several subexpressions, such as "dotask [A-M]*" and then "dotask [N-Z]*", (4) use "xargs" which automatically breaks up long lines for you and runs the command enough times to use all command-line arguments.