Question about pipes in solaris (and others) and buffering.... Post: 40012

Sponsored Content

Operating Systems Solaris Question about pipes in solaris (and others) and buffering.... Post 40012 by sannik on Sunday 7th of September 2003 02:42:31 AM

09-07-2003

Registered User

Question about pipes in solaris (and others) and buffering....

This is weird, so I'm hoping someone here knows solaris and how it handles pipes...

OK... here goes...

Theres this log file, right? I want to tail -f it, grep that, gzip that, then pipe that into more commands. Well thats easy, right?

tail -f file | grep pattern | gzip | otherstuff...

Except...

The output of grep is buffered... Not linebuffered, but 5120 character buffered. As in:

tail -f file | grep pattern | cat

Buffers, and only put stuff to the terminal after grep has outputted 5191 characters.

tail -f file | grep pattern
and
tail -f file | cat

do not.

Using perl, or sed, or tr, or awk, or gzip, or a number of others instead of grep exhibit this behavior as well. It was suggested to me that this might have to do with solaris' bidirectional pipes. I'm not sure that this means.

The buffering doesn't work for me, because with the grep, and further with the gzip, it could take the thing a week to fill once in real use, which isn't really workable for what I need to do.

On another machine, also running solaris, the same thing occurs, except in blocks of 4096. Same on a linux machine.

The question... Is there any way to make a pipe not exhibit this behavior? It is undesirable for my use.

Last edited by sannik; 09-07-2003 at 04:08 AM..

sannik

View Public Profile for sannik

Find all posts by sannik

10 More Discussions You Might Find Interesting

1. Windows & DOS: Issues & Discussions

Windows Buffering during playing

Hello! Will someone out there pls help in clearifying what is really wrong with my system. I use window 98 as my operating system. I am connected to a proxy server for browsing the net. Whenever l am listening to music online l have the problem of intermitted break in which the playing will...

2. Filesystems, Disks and Memory

PIPEs and Named PIPEs (FIFO) Buffer size

Hello! How I can increase or decrease predefined pipe buffer size? System FreeBSD 4.9 and RedHat Linux 9.0 Thanks!

3. UNIX for Advanced & Expert Users

Question on forks and pipes

I am trying to figure out why when i have the following code int main( { printf("0\n"); fork(); printf("1\n"); exit(0);} and type in the shell a.out | cat the output of this program is 0 1 0 1 instead of 0 1 1 does anyone know?

4. Shell Programming and Scripting

Tcl: Outputting Unix console without Buffering

Hi, I'm have a tcl application using expect. The application is connecting to a unix box and is running a script. What i need my app to do is show the output of the console as the script runs in realtime. At the moment, it my app is waiting until the script finishes and then dumps what is...

5. Solaris

A Solaris Question

hi.. I have a question because there are files on Solaris and will not let me change the root user is the owner. excuse is that I come from a Linux environment and I like to learn Solaris thank

6. UNIX for Advanced & Expert Users

Discussion on buffering of standard I/O library

Hi, There are three type of buffering: Fully buffered,line buffered,and unbuffered. The goal of the buffering provided by the standard I/O library is to use the minimum number of read and write calls. I understand that decrease write() calls when use output function output data to a file ,...

7. Programming

Question on pipes in C

If a code forks 2 childs, what can the values be for the process id's of each of the child? I child pid is supposed to be 0, but what if you fork 2 of them?

8. Shell Programming and Scripting

Perl open(CMD, "cmd |"); buffering problem..

Hello, There's a third-party application's command that shows the application's status like "tail -f verybusy.log". When use the command, the output comes every 1-sec. but when it goes in a script below the output comes every 8-sec...What is the problem and how can I fix it? open(CMD,...

9. Solaris

Question with Solaris Crash Analysis Tool with Solaris 9

Hello all. I am new hear and would like to ask a question regarding to the Solaris Crash Analysis Tool. We are analyzing the results of "thread summary" but not quite sure what the asterisk represents. Following are the items that asterisk were attached. 50* threads sleeping on a semaphore (49...

10. Shell Programming and Scripting

Getting piped input from a program that's buffering it's stdout

The old buffering problem again, in a very specific case. On FreeBSD this time, but it's the generic line-buffered vs fully-buffered problem. I'm trying to pick an available bluetooth speaker (all named audio_N), by pinging all of them en mass and taking the first to respond. The...

LEARN ABOUT OPENSOLARIS

install-solaris

install-solaris(1M)                                                                                                            install-solaris(1M)

NAME

       install-solaris - install the Solaris operating system

SYNOPSIS

       install-solaris

       install-solaris  invokes  the  Solaris  Install  program. Depending on graphical capability and available memory at the time of invocation,
       install-solaris invokes either a text-based installer or a graphical installer.

       The following minimum requirements for physical memory dictate which features are available during installation:

       For SPARC machines:

       128 MB

           Minimum physical memory for all installation types

       128 MB

           Minimum physical memory required for windowing system

       384 MB

           Minimum physical memory required for graphical-based installation

       For x86 machines:

       256 MB

           Minimum physical memory for all installation types

       256 MB

           Minimum physical memory required for windowing system

       512 MB

           Minimum physical memory required for graphical-based installation

       In some cases, even if the minimum physical memory is present, available virtual memory after system startup can limit the number  of  fea-
       tures available.

       install-solaris  exists  only  on  the Solaris installation media (CD or DVD) and should be invoked only from there. Refer to the  for more
       details.

       install-solaris allows installation of the operating system onto any standalone system. install-solaris loads the software available on the
       installation media. Refer to the  for disk space requirements.

       Refer to the  for more information on the various menus and selections.

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE         |      ATTRIBUTE VALUE        |
       +-----------------------------+-----------------------------+
       |Availability                 |SUNWcdrom  (Solaris instal-  |
       |                             |lation media)                |
       +-----------------------------+-----------------------------+
       |Interface Stability          |Evolving                     |
       +-----------------------------+-----------------------------+

       pkginfo(1), install(1M), pkgadd(1M), attributes(5)

       It is advisable to exit install-solaris by means of the exit options in the install-solaris menus.

                                                                    23 Sep 2005                                                install-solaris(1M)

10 More Discussions You Might Find Interesting

1. Windows & DOS: Issues & Discussions

Windows Buffering during playing

Discussion started by: kayode

2. Filesystems, Disks and Memory

PIPEs and Named PIPEs (FIFO) Buffer size

Discussion started by: Jus

3. UNIX for Advanced & Expert Users

Question on forks and pipes

Discussion started by: Phantom12345

4. Shell Programming and Scripting

Tcl: Outputting Unix console without Buffering

Discussion started by: Phi01