waiting on jobs in bash, allowing limited parallel jobs at one time, and then for all to finish
Hello,
I am running GNU bash, version 3.2.39(1)-release (x86_64-pc-linux-gnu). I have a specific question pertaining to waiting on jobs run in sub-shells, based on the max number of parallel processes I want to allow, and then wait for the remaining sub-shell jobs to finish before the next step is executed in the pipeline (if I am making proper sense here)..
Essentially,my pseudo code looks like this:
###wait <- works fine without this wait, but I need all kmerlen jobs to finish first to proceed to the next part of the pipeline
## Run the rest of the pipeline...
The first wait statement in the while loop works fine spawning 3 jobs, but when I use the next wait statement, that property is lost, and the number of sub-shells spawned are equal to my kmerlen.
Thanks for any pointers that you can provide.
Last edited by Franklin52; 07-07-2011 at 03:26 AM..
Reason: Please use code tags for code and data samples, thank you
BASH allows you to wait for one specific process via wait PID but doesn't allow you to wait for N random processes. So you'll need to keep a list, and wait in order.
I have multiple jobs and each job dependent on other job.
Each Job generates a log and If job completed successfully log file end's with JOB ENDED SUCCESSFULLY message and if it failed then it will end with JOB ENDED with FAILURE.
I need an help how to start.
Attaching the JOB dependency... (3 Replies)
I have the shell script to call a Perl routine and pass the Informatica WorkFlow name to it. Jobs in each group executes in background do not seem to wait at all. How do I make it to WAIT for the prior group to complete before execute the next group of jobs?
Sample of the jobs flow:
{
... (6 Replies)
I need to process 50 sqlplus scripts which are listed in a text file. I need to develop a shell script that'll read this file and run these sqlplus scripts. At any point of time, the number of sqlplus scripts running shouldn't exceed 6. If any of the sqlplus scripts completes successfully then... (17 Replies)
I have a requirement where jobs/scripts need to be run in the background.The concern here is there are around 20 scripts which need to be run in the bg.Does running all the 20 scripts/job at the same time in bg consumes much sever-utilization. If so wot would be the efficient way to run the jobs... (5 Replies)
Hi,
I'm trying to write a script to decompress a directory full of files. The decompression commands can run in the background, so that many can run at once. But I want to limit the number running at any one time, so that I don't overload the machine.
Something like this:
n=0
for i in *.gz... (15 Replies)
how can i process jobs parallel with conditions below.
Script1.ksh
Script2.ksh
Script3.ksh
Script4.ksh
Script5.ksh
Script6.ksh
Script7.ksh
Script8.ksh
Script9.ksh
Script10.ksh
After successful completion of Script1.ksh I need to run Script7.ksh.
After successful... (4 Replies)
Here's the line I type everyday.
echo "submit daily 00559 00010 00011" | at 21:00
I dunno. I'm totally new to this. Basically I want this to be fool-proof for everyone else we've had people do things like:
subit daily 00559 00010 00011
submit daly 00559 00010 00011
Submit daily (no... (4 Replies)
i need to execute 5 jobs at a time in background and need to get the exit status of all the jobs i wrote small script below , i'm not sure this is right way to do it.any ideas please help.
$cat run_job.ksh
#!/usr/bin/ksh
####################################
typeset -u SCHEMA_NAME=$1
... (1 Reply)
Hi,
Could any one please explain the difference between DataStage server edition jobs and DS parallel extender jobs...?
In which scenarios or application areas do we use either of these jobs.?
Regards
Suresh (0 Replies)