Waiting for an arbitrary background process (limiting number of jobs running)


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Waiting for an arbitrary background process (limiting number of jobs running)
Prev   Next
# 1  
Old 01-28-2010
Waiting for an arbitrary background process (limiting number of jobs running)

Hi,
I'm trying to write a script to decompress a directory full of files. The decompression commands can run in the background, so that many can run at once. But I want to limit the number running at any one time, so that I don't overload the machine.

Something like this:

Code:
n=0
for i in *.gz
do
    gzip -d $i &
    n=$((n+1))
    if [ $n -ge 10 ]; then
        # XXX Not sure what to do here
    fi
done

At the marked spot, I want to wait for one of my background processes to complete. I don't mind which one, but I do want to wait for just one.

wait doesn't work, as it waits for all jobs to complete. On the other hand, wait N doesn't work, because I don't know which job will finish first.

I could use trap "..." 20, but I'd need to be able to pause my script at the XXX line and be able to resume it via the "..." from the trap command. I can't think of a way of doing this ("suspend" in bash might work, but really I need this to work in ksh - I'm not sure the server this will ultimately run on has bash installed).

Can anyone suggest an approach that I could use?

Thanks,
Paul.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Running process in the background

Hi, I have this simple c program that creates duplicate process with fork(): #include <sys/types.h> main() { if (fork() == 0) while(1); else while(1); } I tried running it in the background gcc -o test first.c test & And I got this list of running process: (4 Replies)
Discussion started by: uniran
4 Replies

2. AIX

Use of screen in running background jobs and how to use this

Hello, Please advise use of screen in running jobs in nohup background and how to use this Best regards, Vishal (1 Reply)
Discussion started by: Vishal_dba
1 Replies

3. Shell Programming and Scripting

command to see process running at background

Hi , I want to see all the background process that are running in unix box machine...please guide me is there any specific command for that..since I am executing some scripts at background..!!:confused: (1 Reply)
Discussion started by: nks342
1 Replies

4. Shell Programming and Scripting

waiting on jobs in bash, allowing limited parallel jobs at one time, and then for all to finish

Hello, I am running GNU bash, version 3.2.39(1)-release (x86_64-pc-linux-gnu). I have a specific question pertaining to waiting on jobs run in sub-shells, based on the max number of parallel processes I want to allow, and then wait... (1 Reply)
Discussion started by: srao
1 Replies

5. Shell Programming and Scripting

Waiting for the background process to finish

Hi, I have on shell script which internally calls more than one scripts which run in background. These scripts cannot be modified to run in foreground. eg. myscript.sh -> bulk_launcher.sh -> acq_launcher.sh -> bulk_loader.sh I want the calling shell script myscript.sh to wait till the... (7 Replies)
Discussion started by: AB10
7 Replies

6. Shell Programming and Scripting

Waiting for a background process to finish

Hello, I am a novice shell script programmer. And facing this problem any help is appreciated. I m writing a shell script and running few commands in it background as I have to run them simultaneously. Sample code : sql_prog & sql_prog & sql_prog & echo "Process Completed" Here... (2 Replies)
Discussion started by: vineetbhati
2 Replies

7. Shell Programming and Scripting

How to find the jobs running in background and stop

Hi All, I have requirement. I am running a job every 30mins. before starting the process, i need to check the process, if the process is still running then i need not trigger the process again, if it is not running then trigger the process again. I am using cron to trigger the shell script. Can... (7 Replies)
Discussion started by: srinivas_paluku
7 Replies

8. Shell Programming and Scripting

background jobs exit status and limit the number of jobs to run

i need to execute 5 jobs at a time in background and need to get the exit status of all the jobs i wrote small script below , i'm not sure this is right way to do it.any ideas please help. $cat run_job.ksh #!/usr/bin/ksh #################################### typeset -u SCHEMA_NAME=$1 ... (1 Reply)
Discussion started by: GrepMe
1 Replies

9. Shell Programming and Scripting

Issues with exit after running jobs in background

I have the following sample script to run a script the jobs with the same priority(in this case field3) in parallel; wait for the jobs to finish and run the next set of jobs in parallel.When all the lines are read exit the script. I have the following script which is doing evrything I want... (1 Reply)
Discussion started by: hyennah
1 Replies

10. UNIX for Dummies Questions & Answers

running process in background

I'm trying to install a solaris 9 patch cluster and when I try to use & to run in background it won't allow me to enter in my sudo password so it fails the install and sudo auth. Does Solaris not have screen like linux? If & will work what am I doing wrong? sudo ./install_cluster -q & is... (3 Replies)
Discussion started by: kingdbag
3 Replies
Login or Register to Ask a Question
queuedefs(4)							   File Formats 						      queuedefs(4)

NAME
queuedefs - queue description file for at, batch, and cron SYNOPSIS
/etc/cron.d/queuedefs DESCRIPTION
The queuedefs file describes the characteristics of the queues managed by cron(1M). Each non-comment line in this file describes one queue. The format of the lines are as follows: q.[njobj][nicen][nwaitw] The fields in this line are: q The name of the queue. a is the default queue for jobs started by at(1); b is the default queue for jobs started by batch (see at(1)); c is the default queue for jobs run from a crontab(1) file. njob The maximum number of jobs that can be run simultaneously in that queue; if more than njob jobs are ready to run, only the first njob jobs will be run, and the others will be run as jobs that are currently running terminate. The default value is 100. nice The nice(1) value to give to all jobs in that queue that are not run with a user ID of super-user. The default value is 2. nwait The number of seconds to wait before rescheduling a job that was deferred because more than njob jobs were running in that job's queue, or because the system-wide limit of jobs executing has been reached. The default value is 60. Lines beginning with # are comments, and are ignored. EXAMPLES
Example 1 A sample file. # # a.4j1n b.2j2n90w This file specifies that the a queue, for at jobs, can have up to 4 jobs running simultaneously; those jobs will be run with a nice value of 1. As no nwait value was given, if a job cannot be run because too many other jobs are running cron will wait 60 seconds before trying again to run it. The b queue, for batch(1) jobs, can have up to 2 jobs running simultaneously; those jobs will be run with a nice(1) value of 2. If a job cannot be run because too many other jobs are running, cron(1M) will wait 90 seconds before trying again to run it. All other queues can have up to 100 jobs running simultaneously; they will be run with a nice value of 2, and if a job cannot be run because too many other jobs are running cron will wait 60 seconds before trying again to run it. FILES
/etc/cron.d/queuedefs queue description file for at, batch, and cron. SEE ALSO
at(1), crontab(1), nice(1), cron(1M) SunOS 5.11 1 Mar 1994 queuedefs(4)