Shell Programming and Scripting

Dear World,

I just wrote a script, which puzzled me somewhat. The siginficant code was:

Code:

for file in `ls splits*`; # splits* came from a split command executed earlier
 do
 tail -$SomeNumber $file | cut -d" " -f6 > $file;
done;

The interesting thing is this: A few of the $files were written but most were just empty. They should have contained the sixth field from the cut but didn't.

Is this a bug/feature of bash shells? Is some subprocess faster than another? 42?

Btw.: I fixed it by using some prefix to the $file after the cut redirection. It's somewhat annoying though since execution of the script takes 2 secs longer this way (guess its the filesystem i/o).

Cheers,
BG

Even if the files are empty you're still going to have the overhead of opening reading and closing them, so if it's a lot of files, that may take some time.

Regarding not getting the 6th field. If there is more than one space between each field than using a space as the delimiter to cut can be problematic. If you need to use spaces, I'd suggest running the tail through 'tr' to squeeze out the extra spaces first, and don't put the result back into the same file:

Code:

tail -$SomeNumber $file | tr -s " " | cut -d" " -f6 > ${file}.tmp;

Also, if there is a space at the beginning of the line, your field number will need to increase to '7'. For example, in the lines below:

Code:

field1 field2 field3

 field1 field2 field3

The first line, lines up as you'd expect with a cut using a space as a delimiter. But the second line will have a null field as the first field
making field1 line up with -f2 in cut and field2 line up with -f3, etc.

First off: I don't have a real problem I need to get fixed. And I know my way around on Linux fairly well.

Now: It's not a problem with 'cut' per se. The value I need is actually in the 4th column but since there are two spaces at the begining of ea. line i have to take -f6. No problems here and that's not what was puzzling me!

The thing where my script goes wrong is the redirection after the cut operation. Writing to the same file which was used in the 'tail' doesn't work. Add some arbitrary extension to $file (like you suggested and I already posted) and everything works fine.

I was just wondering where the problem is exactly.
Considering the following flow of command execution:
- tail opens the file, operates on it, closes it (in that order? I don't know)
- the pipe runs the stdout to cut (does this create a subshell???)
- cut takes some column (which contains data and is not empty)
- output redirection to file fails and the resulting files are empty.
BUT: Sometimes (about every 100 times) it does what is expected and the file only contains field 6!

I also tried to debug that line by investigating the output of every command involved. There is no problem except when the last output is redirected to the file.

Again: I don't need help in getting the script to work! Thanks for trying though!

Edit: Ok I tried s.th. here: Instead of redirecting the output to a file with > I did it with 'tee'. After the tee the output is sent to /dev/null. Result: No erronous behaviour as above. Shell script is faster by 2.5 secs (corresponds to 30% in my case). Nice!