Sponsored Content
Top Forums Shell Programming and Scripting Pipes with the < and >> meanings. Post 302518033 by DGPickett on Thursday 28th of April 2011 01:07:44 PM
Old 04-28-2011
Well, it is important to remember that <(...) or >(...) becomes a WORD openable a s a file name, like in ksh ' /dev/fd/7 ', not just a string but with implied separation, so you cannot use it in sed like this, because it looks like 2 arguments. (David Korne says he hadn't thought of it for such usage. You can strip the separation by passing it in a shell subroutine call.):
Code:
sed '
  /xyz/w '>(...)'
  /abc/w '>(...)'
  s/ .*//
 ' infile >outfile

So, some of this is silly, as 'xxx < <(yyy)' is just 'yyy | xxx'. These <() >() constructs are for places where you need a file name but want a pipe, like this:
Code:
xxx | tee >( one-parallel-process-yyy ) | second-parallel-process-zzz

In some UNIX, you have /dev/stdin, /dev/stdout and /dev/stderr, really /dev/fd/0, 1 and 2, which is enough for the occasional obstinate program with no shorthand for stdin/stdout like '-' in tar f, cat, etc. These autoomatically managed named pipes are great if there are more than 2 parallel processes. The bash management of these named pipes is flawed, BTW, bug reported -- they accumulate in /var/tmp. While the ksh in /dev/fd/# UNIX just does a pipe() call and harvests the /dev/fd/# names, the bash implementation uses mknod or mkfifo persistent named pipes, which work a bit different, a pipe() inside open() for one side and connecting to that pipe for the other side, I forget which goes first. For instance, this works to give a c progrqam a robust parallel sort, using the same named pipe twice, for input then for output. Part of the reason this works in the nature of sort, reading all input, sorting and then writing all output:
Code:
system( "rm -rf /tmp/mysort.p ; mknod /tmp/mysort.p p ; sort -o /tmp/mysort.p /tmp/mysort.p &" );
fp = fopen( "/tmp/mysort.p", wb );
while ( . . . ) { . . . ; fputs( data, data_len, fp ); }
fclose( fp );
fp = fopen( "/tmp/mysort.p", rb );
while( fgets( data, data_len, fp )){ . . . }
if ( ferror( fp )){ perror( "sort through pipe failed" ); exit( 1 ); }
fclose( fp );
system( "rm -rf /tmp/mysort.p" );

You can see the overhead of name collision, pipe creation and pipe cleanup, plus if someone else comes along and opens your pipe, it would mess things up. Named pipes were intended for a crude sort of server, where a server process keeps opening the pipe waiting for a client to open the other side the other way, and spinning off a child, somewhat like inetd spinning off server processes for every tcp connection, but it seems tricky to use the fd bidirectionally in a script, without which it is a unidirectional service, like writing a queue.

Some shells consider <() or >() subshells as your login shell's child jobs, with all the start-end notifications and such. Firing up in a subshell means they are not your children, so toss in a protective set of () if you want quiet.

Ksh also has {} to create redirection and inheritance like () but without a fork() to a subshell, but the rules are a bit demanding, so I ignore them. It turns our fork() is 10 times cheaper than execvp(), so for shell speed, count your exec's and use the built-in equivalents like read for line, pattern matching for `grep`, etc. Many nominal PERL scripts are amazingly shellish! Smilie
 

10 More Discussions You Might Find Interesting

1. Filesystems, Disks and Memory

PIPEs and Named PIPEs (FIFO) Buffer size

Hello! How I can increase or decrease predefined pipe buffer size? System FreeBSD 4.9 and RedHat Linux 9.0 Thanks! (1 Reply)
Discussion started by: Jus
1 Replies

2. Shell Programming and Scripting

cd using pipes

Hi, Can the cd command be invoked using pipes??? My actual question is slightly different. I am trying to run an executable from different folders and the path of these folders are obtained dynamically from the front end. Is there a way in which i can actually run the executable... (2 Replies)
Discussion started by: Sinbad
2 Replies

3. Shell Programming and Scripting

where can I get exit code meanings?

I'm investigating strange behaviour on two boxes (Sun OS 5.10 and AIX 5.1) in ksh have used $? to get exit codes returned:- 137 and 34 where can I find what these mean? thank you (1 Reply)
Discussion started by: speedieB
1 Replies

4. Shell Programming and Scripting

named pipes

How to have a conversation between 2 processes using named pipes? (5 Replies)
Discussion started by: kanchan_agr
5 Replies

5. Cybersecurity

Syslog events meanings

Hi everybody, I'm writing to know what the following event stands for. I know that the following event is about a "su to root" action but I don't have any Idea about what action could rise this message. For example If an acction performed by the root crontab, a sudo command or something like that.... (1 Reply)
Discussion started by: PVelazco
1 Replies

6. Shell Programming and Scripting

Pipes not working

Hi, thanks for b4. can anyone tell me why following not working: noUsers=$(who | cut -d" " -f1 | wc -l) What i'm trying to do is get a list of logged on users and pass it to 'wc -l' and store the output to a variable. Any ideas? (1 Reply)
Discussion started by: Furqan_79
1 Replies

7. UNIX for Dummies Questions & Answers

learning about pipes!

im trying to figure out how to do the following: using pipes to combine grep and find commands to print all lines in files that start with the letter f in the current directory that contain the word "test" for example? again using pipes to combine grep and find command, how can I print all... (1 Reply)
Discussion started by: ez45
1 Replies

8. Shell Programming and Scripting

need meanings for FTP codes

Hi Friends, Could i get the meaning for the following FTP codes? 421 425 426 530 450 550 451 551 452 552 553 Thanks, Raja. (1 Reply)
Discussion started by: smr_rashmy
1 Replies

9. Programming

Pipes in C

Hello all, I am trying to learn more about programming Unix pipes in C. I have created a pipe that does od -bc < myfile | head Now, I am trying to create od -bc < myfile | head | wc Here is my code, and I know I might be off, thats why I am here so I can get some clarification. #include... (1 Reply)
Discussion started by: petrca
1 Replies

10. UNIX for Dummies Questions & Answers

top command: abbrevations and meanings - Please !

Hi all, I was trying see some CPU utilization of a Red hat Linux machine using 'top' command. Any way I got high level idea from the out puts, but when I observed the following line: Cpu(s): 7.4%us, 0.5%sy, 0.0%ni, 91.6%id, 0.4%wa, 0.0%hi, 0.1%si, 0.0%st I couldn't make out what... (2 Replies)
Discussion started by: a99u
2 Replies
MAKE_SOCKDFC(8) 					      System Manager's Manual						   MAKE_SOCKDFC(8)

NAME
make_sockdfc - Generates frozen configuratyion file for SOCKS server SYNOPSIS
make_sockdfc [infile [outfile] ] DESCRIPTION
make_sockdfc reads in a plain-text configuration file for the SOCKS server and produces a frozen configuration file as the output. Both arguments are optional. The default for infile is /etc/sockd.conf; the default for outfile is /etc/sockd.fc. You may specify infile while omitting outfile, but you cannot specify outfile without also speficying infile. The contents of the frozen configuration file is essentially the memory image of the parsed input file. Using the frozen configuration file can reduce the start-up delay of the SOCKS server program since it no longer has to parse the file contents. When the SOCKS server starts, it always looks for the frozen configuration file /etc/sockd.fc first. If that file is not found, it then tries to use the plain-text configuration file /etc/sockd.conf. If you use frozen configuration, you must remember to run make_sockdfc every time after you modify the plain-text file or the SOCKS server will continue to use the frozen file of a previous configuration. To find out the contents of a frozen configuration file, use dump_sockdfc. FILES
/etc/sockd.fc, /etc/sockd.conf SEE ALSO
dump_sockdfc(8), sockd.conf(5), sockd.fc(5) AUTHOR
Ying-Da Lee, yingda@best.com or yingda@esd.sgi.com May 6, 1996 MAKE_SOCKDFC(8)
All times are GMT -4. The time now is 08:07 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy