waiting on jobs in bash, allowing limited parallel jobs at one time, and then for all to finish
Hello,
I am running GNU bash, version 3.2.39(1)-release (x86_64-pc-linux-gnu). I have a specific question pertaining to waiting on jobs run in sub-shells, based on the max number of parallel processes I want to allow, and then wait for the remaining sub-shell jobs to finish before the next step is executed in the pipeline (if I am making proper sense here)..
Essentially,my pseudo code looks like this:
###wait <- works fine without this wait, but I need all kmerlen jobs to finish first to proceed to the next part of the pipeline
## Run the rest of the pipeline...
The first wait statement in the while loop works fine spawning 3 jobs, but when I use the next wait statement, that property is lost, and the number of sub-shells spawned are equal to my kmerlen.
Thanks for any pointers that you can provide.
Last edited by Franklin52; 07-07-2011 at 03:26 AM..
Reason: Please use code tags for code and data samples, thank you
Hi,
Could any one please explain the difference between DataStage server edition jobs and DS parallel extender jobs...?
In which scenarios or application areas do we use either of these jobs.?
Regards
Suresh (0 Replies)
i need to execute 5 jobs at a time in background and need to get the exit status of all the jobs i wrote small script below , i'm not sure this is right way to do it.any ideas please help.
$cat run_job.ksh
#!/usr/bin/ksh
####################################
typeset -u SCHEMA_NAME=$1
... (1 Reply)
Here's the line I type everyday.
echo "submit daily 00559 00010 00011" | at 21:00
I dunno. I'm totally new to this. Basically I want this to be fool-proof for everyone else we've had people do things like:
subit daily 00559 00010 00011
submit daly 00559 00010 00011
Submit daily (no... (4 Replies)
how can i process jobs parallel with conditions below.
Script1.ksh
Script2.ksh
Script3.ksh
Script4.ksh
Script5.ksh
Script6.ksh
Script7.ksh
Script8.ksh
Script9.ksh
Script10.ksh
After successful completion of Script1.ksh I need to run Script7.ksh.
After successful... (4 Replies)
Hi,
I'm trying to write a script to decompress a directory full of files. The decompression commands can run in the background, so that many can run at once. But I want to limit the number running at any one time, so that I don't overload the machine.
Something like this:
n=0
for i in *.gz... (15 Replies)
I have a requirement where jobs/scripts need to be run in the background.The concern here is there are around 20 scripts which need to be run in the bg.Does running all the 20 scripts/job at the same time in bg consumes much sever-utilization. If so wot would be the efficient way to run the jobs... (5 Replies)
I need to process 50 sqlplus scripts which are listed in a text file. I need to develop a shell script that'll read this file and run these sqlplus scripts. At any point of time, the number of sqlplus scripts running shouldn't exceed 6. If any of the sqlplus scripts completes successfully then... (17 Replies)
I have the shell script to call a Perl routine and pass the Informatica WorkFlow name to it. Jobs in each group executes in background do not seem to wait at all. How do I make it to WAIT for the prior group to complete before execute the next group of jobs?
Sample of the jobs flow:
{
... (6 Replies)
I have multiple jobs and each job dependent on other job.
Each Job generates a log and If job completed successfully log file end's with JOB ENDED SUCCESSFULLY message and if it failed then it will end with JOB ENDED with FAILURE.
I need an help how to start.
Attaching the JOB dependency... (3 Replies)
Discussion started by: santoshkumarkal
3 Replies
LEARN ABOUT HPUX
queuedefs
queuedefs(4) Kernel Interfaces Manual queuedefs(4)NAME
queuedefs - queue description file for at, batch, and crontab
SYNOPSIS DESCRIPTION
The file describes the characteristics of the queues managed by (see cron(1M)). Each non-comment line in this file describes one queue.
The format of the lines are as follows:
[njob[nice[nwait
The fields in this line are:
q The name of the queue, such that is the default queue for jobs started by (see at(1)), is the queue for jobs started by
(see at(1)), and is the queue for jobs run from a file (see crontab(1)). Queue names through designate user-defined
queues.
njob The maximum number of jobs that can be run simultaneously in that queue. Although any number can be specified here, (see
cron(1M)) by default limits the number of jobs that can be run on all the queues to 100. This limitation can be removed
by setting the variable to 1 in the file.
nice The value to give to all jobs in that queue that are not run with a user ID of super-user (see nice(1)). The default
value is 2.
nwait The number of seconds to wait before rescheduling a job that was deferred because more than njob jobs were running in that
job's queue, or because more than 100 jobs were running in all the queues (see njob above).
EXAMPLES
Consider the following file:
The file is interpreted as follows:
The queue, for jobs (see at(1)), can have up to 4 jobs running simultaneously, and those jobs will be run with a value of
1.
Since no nwait value is given, if a job cannot be run because too many other jobs are running, will wait 60 seconds
before trying again to run it (see cron(1M)).
The queue, for jobs (see at(1)), can have up to 2 jobs running simultaneously. Those jobs will be run with a value of 2.
If a job cannot be run because too many other jobs are running, will wait 90 seconds before trying again to run it.
All other queues can have up to 100 jobs running simultaneously. They will be run with a value of 2, and if a job cannot be run because
too many other jobs are running, will wait 60 seconds before trying again to run it.
SEE ALSO at(1), nice(1), crontab(1), cron(1M), proto(4).
STANDARDS CONFORMANCE queuedefs(4)