Shell Programming and Scripting

It may be that scheduling has the pipe writer going possibly until the written data gets to PIPE_MAX, or it finishes writing. Then grep reads the the whole pipe contents.

That would account for grep producing a burst of output all with one date.
There is no guarantee for every pipe write there is one pipe read

I didn't, but I strongly suspect it is optimisation within grep (perhaps one of the most optimised pieces of software on the planet, given its long history and prevalence).

Another portable, if clunky and inefficient, workaround would be to use a shell while/read loop and run grep on each individual line:

Code:

./input.sh | while read line ; do echo "$line" | grep 2008 ; done | ./output.sh

It seems that many agree it is buffering, either in the pipes or grep. It seems it only happens in particular combinations as well, e.g. if the ./output.sh is ommited grep doesn't buffer. I suspect the shell is a lot more aware of itself than I had considered, especially in choosing where to buffer.

Thank you all for your help. I appreciate the time and have learnt something.

Cheers again!

Mitch.