Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Running jobs in parallel
Top Forums Shell Programming and Scripting Running jobs in parallel Post 302511399 by Corona688 on Wednesday 6th of April 2011 05:41:25 PM
Old 04-06-2011
What's your system? What's your shell?

If you have bash or ksh, you can do something like:

Code:
#!/bin/bash

PROCS=5       ;       WAIT=0  ;       END=0

# Wait for next thread.
function waitnext
{       # Needs BASH/KSH
        wait "${PIDS[$(( (WAIT++) % PROCS))]}"
}

while read LINE
do
        [ "$((END-WAIT))" -ge "$PROCS" ] && waitnext
        stuff "$LINE" &
        PIDS[$(( (END++) % PROCS ))]=$!
done < textfile
# wait for ALL remaining processes
wait

I don't know of a way to make a shell wait for any particular single process though. Either you give it the PID and it waits for that exact process, or you do a general 'wait' and wait for ALL of them. So this has to wait for them in order.

Meaning if one program takes 10x longer than the rest it might end up waiting for that long one before it launches another 5 jobs.
This User Gave Thanks to Corona688 For This Post:
gctex
 

10 More Discussions You Might Find Interesting

1. Programming

running a parallel program

hi , i need to run a parallel program . for example; program1 { array=" the second program should called here : program 2" the execution should continue } the 2nd program should recieve an array of information as argument and it should... (4 Replies)
Discussion started by: bankpro
4 Replies

2. Shell Programming and Scripting

Diff: Server n parallel jobs

Hi, Could any one please explain the difference between DataStage server edition jobs and DS parallel extender jobs...? In which scenarios or application areas do we use either of these jobs.? Regards Suresh (0 Replies)
Discussion started by: sureshg_sampat
0 Replies

3. Shell Programming and Scripting

Running scripts in parallel

Hi, Iam having the scripts as follows. i jus want to run those in parallel. main aim is to minimise the time for overall execution of the script. now out.txt is having 1 lac records. script1(split.sh) split -1000 out.txt splitout ls -A splitout* > filelist.txt cat filelist.txt... (6 Replies)
Discussion started by: nivas
6 Replies

4. IP Networking

running servers parallel

I'm going to undertake a hardware refresh soon and I was wondering if it is possible to run two machines (X and Y) with the same hostname (but different IP addresses) on the same network? Server X is the original server and has an entry in DNS. Server Y is the new server and won't have an entry... (1 Reply)
Discussion started by: soliberus
1 Replies

5. Shell Programming and Scripting

Conditional execution and parallel jobs

how can i process jobs parallel with conditions below. Script1.ksh Script2.ksh Script3.ksh Script4.ksh Script5.ksh Script6.ksh Script7.ksh Script8.ksh Script9.ksh Script10.ksh After successful completion of Script1.ksh I need to run Script7.ksh. After successful... (4 Replies)
Discussion started by: ford2020
4 Replies

6. UNIX for Dummies Questions & Answers

at jobs not running

im doing at now <return> touch ~/Desktop/file.txt <ctrl d> and file is never created though it says job 7 at Wed Mar 17 15:15:55 2010 why? (4 Replies)
Discussion started by: glev2005
4 Replies

7. Shell Programming and Scripting

jobs run parallel - server consumption?

I have a requirement where jobs/scripts need to be run in the background.The concern here is there are around 20 scripts which need to be run in the bg.Does running all the 20 scripts/job at the same time in bg consumes much sever-utilization. If so wot would be the efficient way to run the jobs... (5 Replies)
Discussion started by: michaelrozar17
5 Replies

8. Shell Programming and Scripting

waiting on jobs in bash, allowing limited parallel jobs at one time, and then for all to finish

Hello, I am running GNU bash, version 3.2.39(1)-release (x86_64-pc-linux-gnu). I have a specific question pertaining to waiting on jobs run in sub-shells, based on the max number of parallel processes I want to allow, and then wait... (1 Reply)
Discussion started by: srao
1 Replies

9. UNIX for Dummies Questions & Answers

Running parallel process

i am having 4 process,have to run parallel and not after one by one. sample1.sh sample2.sh sample3.sh sample4.sh Thanks in advance. i (11 Replies)
Discussion started by: sagar_1986
11 Replies

10. Shell Programming and Scripting

Running script in Parallel

Hi Folks I have a doubt. I have a script which is running with 2 input parameters ./GetDSLnkCount.sh <jobnumber> Parmfile.txt I need to run the script in parallel for different jobnumbers. The commands are ./GetDSLnkCount.sh jnhuc14500 Parmfile.txt ./GetDSLnkCount.sh jnhuc14501... (3 Replies)
Discussion started by: morbid_angel
3 Replies

LEARN ABOUT PHP

systemd-networkd-wait-online.service

SYSTEMD-NETWORKD-WAIT-ONLINE.SERVICE(8) 		     systemd-networkd.service			   SYSTEMD-NETWORKD-WAIT-ONLINE.SERVICE(8)

NAME

       systemd-networkd-wait-online.service, systemd-networkd-wait-online - Wait for network to come online

SYNOPSIS

       systemd-networkd-wait-online.service

       /lib/systemd/systemd-networkd-wait-online

DESCRIPTION

       systemd-networkd-wait-online is a one-shot system service that waits for the network to be configured. By default, it will wait for all
       links it is aware of and which are managed by systemd-networkd.service(8) to be fully configured or failed, and for at least one link to
       gain a carrier.

OPTIONS

       The following options are understood:

       -i, --interface=
	   Network interface to wait for before deciding if the system is online. This is useful when a system has several interfaces which will
	   be configured, but a particular one is necessary to access some network resources. This option may be used more than once to wait for
	   multiple network interfaces. When used, all other interfaces are ignored.

       --ignore=
	   Network interfaces to be ignored when deciding if the system is online. By default, only the loopback interface is ignored. This option
	   may be used more than once to ignore multiple network interfaces.

       --timeout=
	   Fail the service if the network is not online by the time the timeout elapses. A timeout of 0 disables the timeout. Defaults to 120
	   seconds.

SEE ALSO

       systemd(1), systemd-networkd.service(8)

systemd 237												   SYSTEMD-NETWORKD-WAIT-ONLINE.SERVICE(8)
All times are GMT -4. The time now is 09:07 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy