Well, the shell is a tool that opens files of various sorts and runs executables. The shell can fork() to create clones processes of itself to run subshells or execvp() programs or scripts. For the pipe between c1 and c2, the shell fork()'s a child that:Voila, two processes running with stdout of c1 pipe()'d to stdin of c2. Of course, it has to do similar steps for the pipe on c2 to c3.

The stdin redirect < is in the wrong place, as c3 has stdin dup()'d to the pipe. The input goes into c1, and the < filename goes after that command's arguments, none in this case. It is after arguments, because, once the shell does any expansion and removes any quotes on arguments, they are the business of execvp() and the command, not the shell. For <, the shell closes stdin (originally your tty, inherited) and then open()'s the filename for read at beginning of file to be stdin.

The open() or dup() will use the lowest number fd not in use, and stdin is 0, stdout is 1. For the overly organized, paranoid and OCD, there is dup2(a,b) to dup to a specific hole in the fd array. An fd, not to be confused with a FILE*, is just an integer to reference a pointer array in the kernel space for each process. The same open file can be pointed to by many processes and many fd in each of those processes, but it is the same open file structure. Web servers even pass open socket fd's to children through a pipe with fancy calls to duplicate the parent open file structure in a child fd. The fd's that all refer to the same open file structure will all be at the same offset (bytes for beginning) for reading or writing. In shell, this means that if one command reads 3 lines and then starts another with stdin inherited, it starts reading at line 4.\

The same filename can be open()'d many times by one or more processes, and these each get new open file structures in the kernel pointing to the same device and inode. This is not the same as many fd pointing to the same open file structure. The open file structures from each open() each have a separate offset pointer for reading or writing the file.

A FILE* is a buffered fd wrapper inside a process that reads ahead or saves output to write bigger blocks. In the shell, if you 'read' a file, it uses a FILE* on stdin and reads ahead, so the next line and a good offset are not available to any inheriting process. To deal with this, you can use command 'line', which read()'s exactly one line, one byte at a time, using a fd or unbuffered FILE*. Every process starts up with three open fd: 0, 1, 2; wrapped in three open FILE* 'stdin', 'stdout', and 'stderr'; stdout is buffered for bulk output, and stderr is unbuffered for error messages that must not be in a buffer if the process crashes out with a fatal signal and perhaps a core file. Wise programmers use or reuse them first before making more fd/FILE*.

Output redirection >> and > are like < but for stdout to write to the filename, possibly creating it; > truncates any old data. More does less, a recurrent theme in shell. For instance, adding '&' means don't wait() for the child to finish. By default, to keep things simple, the shell wait()'s for all the new children to finish before reading more input. Too many & background jobs can exhaust your resources or lock up the system.

A good friend of the pipe is () the subshell. It shares/inherits one stdin, one stdout and one stderr for all commands executed inside it, concatenating output if no '&' or mixing if processes all write in rotation. Further, if there are many background processes and child processes, and you collect all their stderr using "( ... ) 2>&1 | ....", then all processes within must close stderr (exit) before the subshell finishes. This ensures you can see all error messages before proceeding, not missing any from slow, somewhat disconnected child processes.

The reverse of that is tee, which takes one stdin stream and writes a file and stdout the same. Under ksh where there is /dev/fd/# and under bash, you can put redirect to subshell in place of a file name:

Code:

 >( something you can write )
 and
 <( something that writes to you )

.

Pipes are much nicer than temp files, as you never run out of space or have name collisions where someone else owns a file under that name, plus you get 'pipeline parallelism'. For instance, this sorts two files using up to 3 CPUs:

Code:

sort -m <( sort file1 ) <( sort file2 )