Solaris

This is weird, so I'm hoping someone here knows solaris and how it handles pipes...

OK... here goes...

Theres this log file, right? I want to tail -f it, grep that, gzip that, then pipe that into more commands. Well thats easy, right?

tail -f file | grep pattern | gzip | otherstuff...

Except...

The output of grep is buffered... Not linebuffered, but 5120 character buffered. As in:

tail -f file | grep pattern | cat

Buffers, and only put stuff to the terminal after grep has outputted 5191 characters.

tail -f file | grep pattern
and
tail -f file | cat

do not.

Using perl, or sed, or tr, or awk, or gzip, or a number of others instead of grep exhibit this behavior as well. It was suggested to me that this might have to do with solaris' bidirectional pipes. I'm not sure that this means.

The buffering doesn't work for me, because with the grep, and further with the gzip, it could take the thing a week to fill once in real use, which isn't really workable for what I need to do.

On another machine, also running solaris, the same thing occurs, except in blocks of 4096. Same on a linux machine.

The question... Is there any way to make a pipe not exhibit this behavior? It is undesirable for my use.

Huh? Why did this get moved here? It happens on Linux (and probably others) as well.

Anyhoo, I figured it out, apparently pipes are page buffered unless the program itself is line buffered, on linux at least. Some versions of grep and sed have linebuffering options, but the ones on the solaris machine didn't, but you can get the effect by calling select()->flush() after printing a line in perl, so I just did my grepping in that.