Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Two types of pipes?

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Two types of pipes?
# 1  
Old 10-31-2005
Two types of pipes?
What is the difference between:
Code:
cd /tmp
tar -cf - *.txt |gzip > tmp_txt.tar.gz

and
Code:
cd /tmp
mknod pipe p
gzip < pipe > /tmp/tmp_txt1.tar.gz &
tar -cf pipe *.txt

Apart from the fact that we have to create the pipe file manually, is there any difference in the performance of the two? If there is, then which one's better and why?
Last edited by blowtorch; 10-31-2005 at 05:18 PM.. Reason: add line in code
# 2  
Old 10-31-2005
I don't believe there is any huge performance difference.

The ksh code I've seen makes a mknod() call to create a temporary FIFO file as a pipe.
Which is what you're doing to make a named pipe above. The temporary pipe seems to be reused during the life of the processes created.

The only advantage I could see is the idea of I/O leveling - not making a really busy FIFO file on a very busy disk. But I can't see that it would have that much effect.
# 3  
Old 10-31-2005
Your first type of pipe actually is a pipe. That is, it is created by the pipe() system call. That system call returns a pair of fd's in an array. You write stuff on fd[1] and read stuff on fd[0]. Pipes persist across both fork() and exec(). You can wind up with several processes writing to fd[1] or reading from fd[0] (not that multiple readers make that much sense). But all processes using the pipe are desended from the process that issued pipe() in the first place (I'm ignoring a very arcane exception here). Mostly though you have one process writing on fd[1] and another process reading from fd[0]. I just checked the Posix standard, and this is all that the standard officially mandates. Virually every modern version of unix implements an extention: writing on fd[0] and reading from fd[1]. This results in two independent data streams. This emulates the behavior of socketpair() and this extension arose on BSD when pipes were implemented as socket styles objects. (The first implementation was a as a screwy file system object.)

Your second style of pipe was originally called a "named pipe". It is politically incorrect to call them "named pipes" today. (Which is exactly why I call them "named pipes".) You are supposed to call them fifos today. (F. I. F. O == first in, first out). It is not possible for named pipes to be bi-directional like a socketpair() style pipe. There is a single data stream. Very often there will be a single reader and multiple writers. The processes need not be closely related to each other nor to the creator of the named pipe. Permission to read and write is controlled by file system permission. Unlike pipes, named pipes must be opened with open(). open() is expensive relative to pipe().

Which one is faster at transferring data depends on your kernel and how you built it. With HP-UX, for example, you can build a kernel that uses Streams-based pipes, but HP-UX does not support Streams-based fifos. This mostly affects non-standard features like using a zero length write to send a zero length message.
# 4  
Old 10-31-2005
I disagree.
This is what tusc produces in ksh on HPUX 11.00:
Code:
 tusc grep "0" zma.txt | grep "X"
mmap(NULL, 186444, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB|MAP_STATICPREDICTION, 3, 0x9000) .......... = 0xc0010000
mmap(NULL, 14944, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_SHLIB, 3, 0x37000) ................... = 0x705ec000
mmap(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, NULL) .................. = 0x705ea000
mmap(NULL, 1241088, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0x47000) ............................. = 0xc0100000
mmap(NULL, 57344, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS|MAP_SHLIB, -1, NULL) ....... = 0x705dc000
mmap(0x705d3000, 36864, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_SHLIB, 3, 0x176000) .. = 0x705d3000
mmap(NULL, 16384, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, NULL) ................. = 0x705cf000
mmap(NULL, 12288, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0x2000) ................................ = 0xc0003000
mmap(NULL, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_SHLIB, 3, 0x5000) ..................... = 0x705ce000
mmap(NULL, 16384, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, NULL) ................. = 0x705ca000
mmap(NULL, 88, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, NULL) .................... = 0x705c9000
00226764  Bill Print     SEE JOURNAL PAGE FOR PRIVATE LIGHT FIX REQUEST.                                                                                                                                                                                                            Mainframe bill change to handle unassociated lights that are trying to print,.  Lets CSR's see garbage bill via a special bunch co
exit(0) ................................................................................................ WIFEXITED(0)
kcsdev:/home/jmcnama>

No pipe calls. pipe does not call mmap().
# 5  
Old 10-31-2005
Well grep doesn't call pipe() and if it did the ouput from tusc would not contain an "x" in the pipe() line.

Code:
/opt/tusc/bin 379> uname -srm
HP-UX B.11.00 9000/859
/opt/tusc/bin 380> ./tusc ksh -c "/usr/bin/echo hello | grep o" | grep pipe
pipe() ........................................................... = 3 (4)

# 6  
Old 10-31-2005
Quote:
Originally Posted by Perderabo
Your first type of pipe actually is a pipe. That is, it is created by the pipe() system call. That system call returns a pair of fd's in an array. You write stuff on fd[1] and read stuff on fd[0]. Pipes persist across both fork() and exec(). You can wind up with several processes writing to fd[1] or reading from fd[0] (not that multiple readers make that much sense). But all processes using the pipe are desended from the process that issued pipe() in the first place (I'm ignoring a very arcane exception here). Mostly though you have one process writing on fd[1] and another process reading from fd[0]. I just checked the Posix standard, and this is all that the standard officially mandates. Virually every modern version of unix implements an extention: writing on fd[0] and reading from fd[1]. This results in two independent data streams. This emulates the behavior of socketpair() and this extension arose on BSD when pipes were implemented as socket styles objects. (The first implementation was a as a screwy file system object.)

Your second style of pipe was originally called a "named pipe". It is politically incorrect to call them "named pipes" today. (Which is exactly why I call them "named pipes".) You are supposed to call them fifos today. (F. I. F. O == first in, first out). It is not possible for named pipes to be bi-directional like a socketpair() style pipe. There is a single data stream. Very often there will be a single reader and multiple writers. The processes need not be closely related to each other nor to the creator of the named pipe. Permission to read and write is controlled by file system permission. Unlike pipes, named pipes must be opened with open(). open() is expensive relative to pipe().

Which one is faster at transferring data depends on your kernel and how you built it. With HP-UX, for example, you can build a kernel that uses Streams-based pipes, but HP-UX does not support Streams-based fifos. This mostly affects non-standard features like using a zero length write to send a zero length message.
Oh yeah, --slap forehead--, this is basic IPC! Its just that was really getting confused with these over the last couple of days, and just had to find out what exactly the difference was performance-wise (I always use the '|' to link I/O of different processes).
 
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Programming

Are These All The Integer Types In C

Hello and Good day, I am currently studying C and I just finished learning about variables mainly those of integer type. I am wondering if the list below are all there is to integer variables and there are still more that i have to learn. Here are the list: Char Short int long long long... (3 Replies)
Discussion started by: split_func0
3 Replies

2. Shell Programming and Scripting

Cp -r except certain file types

the following excludes certain directories successfully cp -r probe/!(dir) /destination I want to exclude certain file types and tried unsuccessfully cp -r probe/!(*.avi) /destination (2 Replies)
Discussion started by: tmf
2 Replies

3. Homework & Coursework Questions

Help with using different types of GREP

1. The problem statement, all variables and given/known data: Hey there, I'm brand new to using Unix as I just started a course on it in my University, and I currently working through a worksheet which focuses on the many commands and methods of GREP (I'm working through the terminal command... (11 Replies)
Discussion started by: SilvarHawke
11 Replies

4. What is on Your Mind?

What Types of Food Do You Like The Most?

On another simple topic, multiple choice answers OK ( you can pick more than one or suggest others - we will add your suggestions to the poll ). What Types of Food Do You Like The Most? (27 Replies)
Discussion started by: Neo
27 Replies

5. UNIX for Dummies Questions & Answers

mime types

Hi, I am trying to launch an ogg movie from a pdf file which has been produced with pdflatex and \movie {\centerline{\includegraphics {grafiques_xerrades/un_manolo_amb_camera.pdf}}} {hlims_xerrades/XocCumuls.ogg} The switch "externalviewer" makes kpdf launch the default... (5 Replies)
Discussion started by: pau
5 Replies

6. Filesystems, Disks and Memory

PIPEs and Named PIPEs (FIFO) Buffer size

Hello! How I can increase or decrease predefined pipe buffer size? System FreeBSD 4.9 and RedHat Linux 9.0 Thanks! (1 Reply)
Discussion started by: Jus
1 Replies

7. Filesystems, Disks and Memory

associated file types

I have a file of type .for extension .In a guui based unix environment like solaris if I double click on that file a specific program designed by me has to run which takes this file as the parameter and exceutes the program. Can anyone help me? (8 Replies)
Discussion started by: nhk_srd
8 Replies

8. UNIX for Dummies Questions & Answers

backup types

can anyone explain me difference between tar and ufsdump commands.............and also i wd like to know the difference between incremental and differential backup......... thx in advance (1 Reply)
Discussion started by: girish_shukla
1 Replies
Login or Register to Ask a Question