Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Doubt about pipes and subprocess

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Doubt about pipes and subprocess
# 1  
Old 01-19-2010
Doubt about pipes and subprocess
Hi,

I am having a trivial doubt. Please see the below pipeline code sequence.

Code:
command1 | (command 2; commend 3)

I am aware that the command that follows pipe will run in the sub shell by the Unix kernel. But how about here? Since these set of commands are grouped under "parantheses", will they run inside another sub shell of pipe's shell? Hope my question is not so be-wilder to be answered Smilie
Last edited by royalibrahim; 01-20-2010 at 02:33 PM..
# 2  
Old 01-21-2010
I would like to know, whether I have asked a sensible question because I have not got any answer for it.
# 3  
Old 01-21-2010
We're not ignoring you if we haven't answered the instant you want an answer. You don't have to bump threads. In fact you shouldn't, it's against the rules you read and agreed to when you registered.

I don't think you've got that quite right. Without complex structures like brackets or loops, a command after a pipe needs no subshell. The shell just creates a new process, arranges the files how you wanted, and replaces the new process with the requested command.

A subshell does have to stick around to manage situations like you illustrated:
Code:
command1 | ( command2 ; command3 )

It has to stay resident in order to wait for command2 to finish before starting command3.

A subshell also exists in a situation like this:

Code:
command1 | while read LINE
do
        echo "${LINE}"
done

The subshell has to stay resident to process code after the pipe inside the loop.

These subshells have side-effects. Try this:

Code:
VARIABLE="hello"
echo asdf | ( cat ; VARIABLE="goodbye" )

echo "VARIABLE=${VARIABLE}"

It will output "hello", since variables changed inside the subshell aren't changed in its parent.
# 4  
Old 01-21-2010
Hope this helps:

Code:
bash-3.2$ cat getppid.c 

#include <stdio.h>
#include <unistd.h>

int main ()
{
  fprintf (stdout, "%d\n", getppid ());
  return 0;
}

bash-3.2$ make getppid
cc     getppid.c   -o getppid

bash-3.2$ ./getppid >> t |  ( ./getppid >> t ; ./getppid >> t ) ; cat t
14236
14774
14774

I assume you talk about bash - tcsh is different here:

Code:
tcsh $  ( ./getppid ; ./getppid )
14867
12872

Last edited by Andre_Merzky; 01-21-2010 at 03:59 PM.. Reason: Who needs preview if you can edit ;-)
# 5  
Old 01-21-2010
Thank you Corona688 for your reply.

But, anything to the right of a pipe will run in a subshell, not just loops. In non POSIX shells, the pipe spawns the sub shell. The use of a single pipe in a shell creates TWO subshells, one for each side (a pipeline with two | creates three subshells, for the three commands, etc.)

So my question is, how many subshells would've spawned in this scenario.

Also, in
Code:
 (command &)

The background job detaches from the current shell and runs in a subshell (separate environment) and also, we are encapsulating this command inside parantheses, hence we are forcing it to run inside a subshell again. So would it create 2 subshells here? one for parantheses and one for "background"??
# 6  
Old 01-21-2010
Quote:
Originally Posted by royalibrahim
Thank you Corona688 for your reply.

But, anything to the right of a pipe will run in a subshell, not just loops
Code:
$ sleep 9000 | cat &
[1] 7557
$ ps
  PID TTY          TIME CMD
 7543 pts/3    00:00:00 bash
 7556 pts/3    00:00:00 sleep
 7557 pts/3    00:00:00 cat
 7559 pts/3    00:00:00 ps

There is no subshell.
Code:
$ sleep 9000 | cat | cat | cat | cat | cat &
[1] 7581
$ ps
  PID TTY          TIME CMD
 7543 pts/3    00:00:00 bash
 7576 pts/3    00:00:00 sleep
 7577 pts/3    00:00:00 cat
 7578 pts/3    00:00:00 cat
 7579 pts/3    00:00:00 cat
 7580 pts/3    00:00:00 cat
 7581 pts/3    00:00:00 cat
 7584 pts/3    00:00:00 ps

There's not 5 subshells either.
Quote:
So my question is, how many subshells would've spawned in this scenario.
I think you're under a misapprehension here. The shell does clone itself in order to do redirection through pipes, but unless its actually needed, the subshell does not stick around: Once it's redirected file descriptors the way you asked it to, it replaces itself with the program you asked it to run. By the very act of running the process you asked it to, the subshell is wiped out. In its place is a brand new process with all the same redirections as the subshell used to have.
Quote:
The background job detaches from the current shell and runs in a subshell (separate environment) and also, we are encapsulating this command inside parantheses
Not "also". A subshell remains because you encapsulated it with brackets. If you had not, no subshell would remain -- the subshell would exist for less than eyeblink, connecting file descriptors as specified then replacing itself with the new command. Why bother waiting around when there's nothing left for it to do?

...I'm beginning to have a sneaking suspicion that when you say "subshell" you mean "process". Not all processes are shells.
# 7  
Old 01-21-2010
Expanding a bit on the good explanations above:

When you ask the shell to run the "cat" command the shell calls one of the exec functions:
Quote:
int execl(const char *path, const char *arg0, ..., const
char *argn, char * /*NULL*/);

int execv(const char *path, char *const argv[]);

int execle(const char *path, const char *arg0, ..., const
char *argn, char * /*NULL*/, char *const envp[]);

int execve(const char *path, char *const argv[], char *const
envp[]);

int execlp(const char *file, const char *arg0, ..., const
char *argn, char * /*NULL*/);

int execvp(const char *file, char *const argv[]);
The result is the cat executable is then running in the new process, not the shell.
When the process ends, the shell resumes in the parent process which waited for the cat process to end.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

[Python] - subprocess issue

So I have this basic script, see below import subprocess import shlex command = "gcloud projects list" subprocess.check_output(shlex.split(command)) subprocess.check_call(shlex.split(command)) The subprocess.check_call(shlex.split(command)) actually return what I expect. It returns... (6 Replies)
Discussion started by: scj2012
6 Replies

2. Shell Programming and Scripting

Python3 subprocess troubles

After struggling with this for days now, I'm reaching out to the experts of all things linux for some help with this. I'm trying to run the following working command (on command line) inside a python script using subprocess: rsync -avzh --no-perms --delete --include="*sub*" --exclude='*'... (2 Replies)
Discussion started by: timj123
2 Replies

3. Shell Programming and Scripting

Python subprocess

Hi guys, I'm learning python and perl and i was trying to run from python a perl script using the subprocess module. I have an issue that i don't understand regarding this. I run this code: #!/usr/bin/python import subprocess p2 = subprocess.Popen(,stdout=subprocess.PIPE) output2 =... (2 Replies)
Discussion started by: capitanui
2 Replies

4. Shell Programming and Scripting

Xterm using python subprocess

Hi, I am trying to run a shell script using subprocess in python. I can run simple script with arguments using subprocess.But I am not able to embed xterm in subrocess command. #!/usr/bin/python import subprocess subprocess.call() Above code gives me error. Please help me in... (2 Replies)
Discussion started by: diehard
2 Replies

5. Shell Programming and Scripting

Python subprocess module

I need to run this command using python subprocess module (notice I'm using only variables): cmd = TESTPATH + ' -s ' + serviceName + ' -r ' + rdir + \ ' -m ' + masterAcct + ' -p ' + persona + ' -P ' + passwd (3 Replies)
Discussion started by: erick_tuk
3 Replies

6. Shell Programming and Scripting

python: what's wrong with my subprocess.Popen

my script is #!/usr/bin/env python import datetime import subprocess import sys import os import signal from time import sleep def runForAWhile(cmd, secs=10): print("running %s" % cmd) timeout = datetime.timedelta(seconds=secs) print timeout proc = subprocess.Popen(cmd,... (0 Replies)
Discussion started by: yanglei_fage
0 Replies

7. AIX

Subprocess errors

Hi Guys, Just a question about subprocesses.. Lately one of our servers has started to throw out the following error: SYSTEM ERROR: Too many subprocesses, cannot fork. Errno=12 We've already increased the threshold twice. Its now up to 8000 and the swap space has also been increased. We... (6 Replies)
Discussion started by: Jazmania
6 Replies

8. Shell Programming and Scripting

How to wait for the subprocess to finish in tcl

Hi All Here i have a piece of code, set filename "./GopiRun.sh" #I need to wait here until the GopiRun.sh is completed how do i achive this exit. (1 Reply)
Discussion started by: nathgopi214
1 Replies

9. Shell Programming and Scripting

doubt about pipes

can we use pipes to redirect the output of any command to grep ..... like i wanted to write this script about checking the online status of a certain user so ...can i send the output of who to grep directly using pipes... one way was this : who > temp grep $uname temp i was wondering if... (4 Replies)
Discussion started by: evergreen_cool
4 Replies

10. Filesystems, Disks and Memory

PIPEs and Named PIPEs (FIFO) Buffer size

Hello! How I can increase or decrease predefined pipe buffer size? System FreeBSD 4.9 and RedHat Linux 9.0 Thanks! (1 Reply)
Discussion started by: Jus
1 Replies
Login or Register to Ask a Question