Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How to make sure that sub shells have finished before moving on?
Top Forums Shell Programming and Scripting How to make sure that sub shells have finished before moving on? Post 302845017 by LMHmedchem on Tuesday 20th of August 2013 02:51:53 PM
Old 08-20-2013
How to make sure that sub shells have finished before moving on?
Hello,

I have a script that is running multiple instances of an application in parallel.

Code:
# learn on f0
emergent -nogui -p $ScriptLoc/$PROJ fold_tag=f0 &
sleep 5

# learn on f1
emergent -nogui -p $ScriptLoc/$PROJ fold_tag=f1 &
sleep 5

# learn on f2
emergent -nogui -p $ScriptLoc/$PROJ fold_tag=f2 &
sleep 5

# learn on f3
emergent -nogui -p $ScriptLoc/$PROJ fold_tag=f3 &
sleep 5

# learn on f4
emergent -nogui -p $ScriptLoc/$PROJ fold_tag=f4

# wait 4 minutes to make sure that processing has finished for all folds
sleep 240

echo "statistics"  
./stats4A_04.sh $BATCH 0 &
./stats4A_04.sh $BATCH 1 &
./stats4A_04.sh $BATCH 2 &
./stats4A_04.sh $BATCH 3 &
./stats4A_04.sh $BATCH 4

There are short sleep instances between where each instance is called. This helps keep the output to the terminal from getting garbled as multiple programs are printing at the same time. When all the instances have returned a separate program is called to process the output.

The problem I am having is that the output processing app (stats4A) is getting called before all of the instances have finished. I think that if f4 finishes before the others, then the script proceeds to call the stats program.

This more or less calls for a mutex to keep the stats program from running until everything has finished. There is no way to guarantee that f4 will finish after the others. You can see that I am already waiting 4 minutes as a measure to address this, but it appears that does not always work.

The only thing I can think of at the moment would be to have a 0/1 variable for each instance. That variable would initialize to 0. I would call the instances in a function, instead of inline, and have the function set the variable to 1 after the instance has returned. Then I would need the stats calls to be in something like a while loop that would not call stats until all of the values were 1. In other words, something needs to loop until all the values are 1 and then move on to the stats calls.

This would be relatively easy in cpp, but I really have no idea in bash. Is there a standard way to do this in a shell script? I can't imagine I am the first person to run across this.

Suggestions would be appreciated.

LMHmedchem
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Finding Out When A Process Has Finished?

Problem I have an application which basically runs lots of UNIX programs remotely, using the Telnet protocol. For each program it remotely executes, it stores the process ID (PID) for that process. At regular intervals, I would like my application to take the PID for every process still... (5 Replies)
Discussion started by: 1cuervo
5 Replies

2. UNIX for Dummies Questions & Answers

Has my script finished?

Hi I'm writing a script which will be run by cron every X minutes. I don't want cron to run my script again if the previous one has not yet finished. When the script first runs, I had the idea to store the Process ID in a file. When cron tries to run the script again, I would check the... (5 Replies)
Discussion started by: Bab00shka
5 Replies

3. Shell Programming and Scripting

** Finished ** Syncid.rc

So, the script I've been working on, since I was starting to learn Shell scripting is now complete. This was coded in ksh, and I am very proud of it. What this script does, is syncs up uid's across the network. So if you have 10 servers, with 10 usernames with different UID's - this will... (1 Reply)
Discussion started by: syndex
1 Replies

4. Shell Programming and Scripting

How can I trigger another make command when one is finished?

Hello all I have to run manually make commands in our system the make compilations task's takes very long And I like to be able to run another make task right after one is finished. What is the best way to automate it ? (2 Replies)
Discussion started by: umen
2 Replies

5. Shell Programming and Scripting

How to know a executable has finished his task

Hi frnds, I want to know is there a way by which we can know that a C++ executable has finished its job in shell script. My task is as follows: 1.Shell script calls a executable 2.Executable executes and performs its job of generating some reports. Now i want my shell script to... (4 Replies)
Discussion started by: electroon
4 Replies

6. Shell Programming and Scripting

How do I know all processes are finished?

Hi all, I am writing a C shell script that starts a program. The program forks of several child processes. Only when all child processes are done, I want to archive my log files. Below is what I have so far, but unfortunately it doesn't work. MyProgram if (-e processes.txt) then rm... (2 Replies)
Discussion started by: Carla
2 Replies

7. UNIX for Advanced & Expert Users

how to make a parent wait on a child shells running in background?

Hi I have a shell script A which calls another 10 shell scripts which run in background. How do i make the parent script wait for the child scripts complete, or in other words, i must be able to do a grep of parent script to find out if the child scripts are still running. My Code: ... (1 Reply)
Discussion started by: albertashish
1 Replies

8. Programming

How can I tell when recv is finished with pending data?

I'm working with recv and I am having a heck of a lot of trouble ignoring excess data that I did not ask for. I was hoping someone could shine some light on the subject for me because I'm not getting anywhere fast. ---------- Post updated at 02:46 AM ---------- Previous update was at 12:31 AM... (2 Replies)
Discussion started by: Errigour
2 Replies

9. Shell Programming and Scripting

how to proceed when curl is finished

I have a script which uses cli curl to download the source code of a webpage and then tests if a specific string exists in the source. The problem is that the website has a slow response, so the eval expression hasn't completed when the test starts. The test returns a negative, and the curl... (8 Replies)
Discussion started by: locoroco
8 Replies

10. Shell Programming and Scripting

Wait until firefox has finished

I am running a macro script from the command line. But the script doesn't wait until the task has finished. firefox imacros://run/?m=macro_script.iim firefox imacros://run/?m=macro_script2.iim How do I get it to wait until the macro has been completed? I am using imacros, a firefox... (1 Reply)
Discussion started by: locoroco
1 Replies

LEARN ABOUT SUNOS

startpar

STARTPAR(8)						      System Manager's Manual						       STARTPAR(8)

NAME

       startpar - start runlevel scripts in parallel

SYNOPSIS

       startpar [-p par] [-i iorate] [-t timeout] [-T global_timeout] [-a arg] prg1 prg2 ...
       startpar [-p par] [-i iorate] [-t timeout] [-T global_timeout] -M [ boot|start|stop]

DESCRIPTION

       startpar  is  used  to run multiple run-level scripts in parallel.  The degree of parallelism on one CPU can be set with the -p option, the
       default is full parallelism. An argument to all of the scripts can be provided with the -a option.  Processes blocked by pending  I/O  will
       cause  new  process  creation  to be weighted by the iorate factor 800.	To change this factor the option -i can be used to specify another
       value.  The amount weight=(nblockedxiorate)/1000 will be subtracted from the total number  of  processes  which	could  be  started,  where
       nblocked is the number of processes currently blocked by pending I/O.

       The  output  of	each  script is buffered and written when the script exits, so output lines of different scripts won't mix. You can modify
       this behaviour by setting a timeout.

       The timeout set with the -t option is used as buffer timeout. If the output buffer of a script is not empty and the last output was timeout
       seconds ago, startpar will flush the buffer.

       The  -T option timeout works more globally. If no output is printed for more than global_timeout seconds, startpar will flush the buffer of
       the script with the oldest output. Afterwards it will only print output of this script until it is finished.

       The -M option switches startpar into a make(1) like behaviour.  This option takes three different arguments:  boot,  start,  and  stop  for
       reading .depend.boot or .depend.start or .depend.stop respectively in the directory /etc/init.d/.  By scanning the boot and runlevel direc-
       tories in /etc/init.d/ it then executes the appropriate scripts in parallel.

FILES

       /etc/init.d/.depend.boot
       /etc/init.d/.depend.start
       /etc/init.d/.depend.stop

SEE ALSO

       init(8) insserv(8).

COPYRIGHT

       2003,2004 SuSE Linux AG, Nuernberg, Germany.
       2007 SuSE LINUX Products GmbH, Nuernberg, Germany.

AUTHOR

       Michael Schroeder <mls@suse.de>
       Takashi Iwai <tiwai@suse.de>
       Werner Fink <werner@suse.de>

								     Jun 2003							       STARTPAR(8)
All times are GMT -4. The time now is 04:46 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy