The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > UNIX for Dummies Questions & Answers
.
google unix.com



UNIX for Dummies Questions & Answers If you're not sure where to post a UNIX or Linux question, post it here. All UNIX and Linux newbies welcome !!

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
What Types of Food Do You Like The Most? Neo What's on Your Mind? 27 10-06-2009 12:44 AM
PIPEs and Named PIPEs (FIFO) Buffer size Jus Filesystems, Disks and Memory 1 08-20-2004 11:14 AM
associated file types nhk_srd Filesystems, Disks and Memory 8 06-11-2004 05:17 PM
backup types girish_shukla UNIX for Dummies Questions & Answers 1 11-01-2003 04:36 PM
So many types of LINUX's! Minnesota Red UNIX for Dummies Questions & Answers 2 06-29-2003 09:28 PM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 10-31-2005
blowtorch's Avatar
blowtorch blowtorch is offline Forum Advisor  
Supporter
  
 

Join Date: Dec 2004
Location: Singapore
Posts: 2,350
Two types of pipes?

What is the difference between:

Code:
cd /tmp
tar -cf - *.txt |gzip > tmp_txt.tar.gz

and

Code:
cd /tmp
mknod pipe p
gzip < pipe > /tmp/tmp_txt1.tar.gz &
tar -cf pipe *.txt

Apart from the fact that we have to create the pipe file manually, is there any difference in the performance of the two? If there is, then which one's better and why?

Last edited by blowtorch; 10-31-2005 at 05:18 PM.. Reason: add line in code
  #2 (permalink)  
Old 10-31-2005
jim mcnamara jim mcnamara is offline Forum Staff  
...@...
  
 

Join Date: Feb 2004
Location: NM
Posts: 5,807
I don't believe there is any huge performance difference.

The ksh code I've seen makes a mknod() call to create a temporary FIFO file as a pipe.
Which is what you're doing to make a named pipe above. The temporary pipe seems to be reused during the life of the processes created.

The only advantage I could see is the idea of I/O leveling - not making a really busy FIFO file on a very busy disk. But I can't see that it would have that much effect.
  #3 (permalink)  
Old 10-31-2005
Perderabo's Avatar
Perderabo Perderabo is offline Forum Staff  
Unix Daemon
  
 

Join Date: Aug 2001
Location: Ashburn, Virginia
Posts: 9,131
Your first type of pipe actually is a pipe. That is, it is created by the pipe() system call. That system call returns a pair of fd's in an array. You write stuff on fd[1] and read stuff on fd[0]. Pipes persist across both fork() and exec(). You can wind up with several processes writing to fd[1] or reading from fd[0] (not that multiple readers make that much sense). But all processes using the pipe are desended from the process that issued pipe() in the first place (I'm ignoring a very arcane exception here). Mostly though you have one process writing on fd[1] and another process reading from fd[0]. I just checked the Posix standard, and this is all that the standard officially mandates. Virually every modern version of unix implements an extention: writing on fd[0] and reading from fd[1]. This results in two independent data streams. This emulates the behavior of socketpair() and this extension arose on BSD when pipes were implemented as socket styles objects. (The first implementation was a as a screwy file system object.)

Your second style of pipe was originally called a "named pipe". It is politically incorrect to call them "named pipes" today. (Which is exactly why I call them "named pipes".) You are supposed to call them fifos today. (F. I. F. O == first in, first out). It is not possible for named pipes to be bi-directional like a socketpair() style pipe. There is a single data stream. Very often there will be a single reader and multiple writers. The processes need not be closely related to each other nor to the creator of the named pipe. Permission to read and write is controlled by file system permission. Unlike pipes, named pipes must be opened with open(). open() is expensive relative to pipe().

Which one is faster at transferring data depends on your kernel and how you built it. With HP-UX, for example, you can build a kernel that uses Streams-based pipes, but HP-UX does not support Streams-based fifos. This mostly affects non-standard features like using a zero length write to send a zero length message.
  #4 (permalink)  
Old 10-31-2005
jim mcnamara jim mcnamara is offline Forum Staff  
...@...
  
 

Join Date: Feb 2004
Location: NM
Posts: 5,807
I disagree.
This is what tusc produces in ksh on HPUX 11.00:

Code:
 tusc grep "0" zma.txt | grep "X"
mmap(NULL, 186444, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB|MAP_STATICPREDICTION, 3, 0x9000) .......... = 0xc0010000
mmap(NULL, 14944, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_SHLIB, 3, 0x37000) ................... = 0x705ec000
mmap(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, NULL) .................. = 0x705ea000
mmap(NULL, 1241088, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0x47000) ............................. = 0xc0100000
mmap(NULL, 57344, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS|MAP_SHLIB, -1, NULL) ....... = 0x705dc000
mmap(0x705d3000, 36864, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_SHLIB, 3, 0x176000) .. = 0x705d3000
mmap(NULL, 16384, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, NULL) ................. = 0x705cf000
mmap(NULL, 12288, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0x2000) ................................ = 0xc0003000
mmap(NULL, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_SHLIB, 3, 0x5000) ..................... = 0x705ce000
mmap(NULL, 16384, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, NULL) ................. = 0x705ca000
mmap(NULL, 88, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, NULL) .................... = 0x705c9000
00226764  Bill Print     SEE JOURNAL PAGE FOR PRIVATE LIGHT FIX REQUEST.                                                                                                                                                                                                            Mainframe bill change to handle unassociated lights that are trying to print,.  Lets CSR's see garbage bill via a special bunch co
exit(0) ................................................................................................ WIFEXITED(0)
kcsdev:/home/jmcnama>

No pipe calls. pipe does not call mmap().
  #5 (permalink)  
Old 10-31-2005
Perderabo's Avatar
Perderabo Perderabo is offline Forum Staff  
Unix Daemon
  
 

Join Date: Aug 2001
Location: Ashburn, Virginia
Posts: 9,131
Well grep doesn't call pipe() and if it did the ouput from tusc would not contain an "x" in the pipe() line.


Code:
/opt/tusc/bin 379> uname -srm
HP-UX B.11.00 9000/859
/opt/tusc/bin 380> ./tusc ksh -c "/usr/bin/echo hello | grep o" | grep pipe
pipe() ........................................................... = 3 (4)

  #6 (permalink)  
Old 10-31-2005
blowtorch's Avatar
blowtorch blowtorch is offline Forum Advisor  
Supporter
  
 

Join Date: Dec 2004
Location: Singapore
Posts: 2,350
Quote:
Originally Posted by Perderabo
Your first type of pipe actually is a pipe. That is, it is created by the pipe() system call. That system call returns a pair of fd's in an array. You write stuff on fd[1] and read stuff on fd[0]. Pipes persist across both fork() and exec(). You can wind up with several processes writing to fd[1] or reading from fd[0] (not that multiple readers make that much sense). But all processes using the pipe are desended from the process that issued pipe() in the first place (I'm ignoring a very arcane exception here). Mostly though you have one process writing on fd[1] and another process reading from fd[0]. I just checked the Posix standard, and this is all that the standard officially mandates. Virually every modern version of unix implements an extention: writing on fd[0] and reading from fd[1]. This results in two independent data streams. This emulates the behavior of socketpair() and this extension arose on BSD when pipes were implemented as socket styles objects. (The first implementation was a as a screwy file system object.)

Your second style of pipe was originally called a "named pipe". It is politically incorrect to call them "named pipes" today. (Which is exactly why I call them "named pipes".) You are supposed to call them fifos today. (F. I. F. O == first in, first out). It is not possible for named pipes to be bi-directional like a socketpair() style pipe. There is a single data stream. Very often there will be a single reader and multiple writers. The processes need not be closely related to each other nor to the creator of the named pipe. Permission to read and write is controlled by file system permission. Unlike pipes, named pipes must be opened with open(). open() is expensive relative to pipe().

Which one is faster at transferring data depends on your kernel and how you built it. With HP-UX, for example, you can build a kernel that uses Streams-based pipes, but HP-UX does not support Streams-based fifos. This mostly affects non-standard features like using a zero length write to send a zero length message.
Oh yeah, --slap forehead--, this is basic IPC! Its just that was really getting confused with these over the last couple of days, and just had to find out what exactly the difference was performance-wise (I always use the '|' to link I/O of different processes).
Closed Thread

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 01:05 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0