Sponsored Content
Full Discussion: Running jobs in parallel
Top Forums Shell Programming and Scripting Running jobs in parallel Post 302511399 by Corona688 on Wednesday 6th of April 2011 05:41:25 PM
Old 04-06-2011
What's your system? What's your shell?

If you have bash or ksh, you can do something like:

Code:
#!/bin/bash

PROCS=5       ;       WAIT=0  ;       END=0

# Wait for next thread.
function waitnext
{       # Needs BASH/KSH
        wait "${PIDS[$(( (WAIT++) % PROCS))]}"
}

while read LINE
do
        [ "$((END-WAIT))" -ge "$PROCS" ] && waitnext
        stuff "$LINE" &
        PIDS[$(( (END++) % PROCS ))]=$!
done < textfile
# wait for ALL remaining processes
wait

I don't know of a way to make a shell wait for any particular single process though. Either you give it the PID and it waits for that exact process, or you do a general 'wait' and wait for ALL of them. So this has to wait for them in order.

Meaning if one program takes 10x longer than the rest it might end up waiting for that long one before it launches another 5 jobs.
This User Gave Thanks to Corona688 For This Post:
 

10 More Discussions You Might Find Interesting

1. Programming

running a parallel program

hi , i need to run a parallel program . for example; program1 { array=" the second program should called here : program 2" the execution should continue } the 2nd program should recieve an array of information as argument and it should... (4 Replies)
Discussion started by: bankpro
4 Replies

2. Shell Programming and Scripting

Diff: Server n parallel jobs

Hi, Could any one please explain the difference between DataStage server edition jobs and DS parallel extender jobs...? In which scenarios or application areas do we use either of these jobs.? Regards Suresh (0 Replies)
Discussion started by: sureshg_sampat
0 Replies

3. Shell Programming and Scripting

Running scripts in parallel

Hi, Iam having the scripts as follows. i jus want to run those in parallel. main aim is to minimise the time for overall execution of the script. now out.txt is having 1 lac records. script1(split.sh) split -1000 out.txt splitout ls -A splitout* > filelist.txt cat filelist.txt... (6 Replies)
Discussion started by: nivas
6 Replies

4. IP Networking

running servers parallel

I'm going to undertake a hardware refresh soon and I was wondering if it is possible to run two machines (X and Y) with the same hostname (but different IP addresses) on the same network? Server X is the original server and has an entry in DNS. Server Y is the new server and won't have an entry... (1 Reply)
Discussion started by: soliberus
1 Replies

5. Shell Programming and Scripting

Conditional execution and parallel jobs

how can i process jobs parallel with conditions below. Script1.ksh Script2.ksh Script3.ksh Script4.ksh Script5.ksh Script6.ksh Script7.ksh Script8.ksh Script9.ksh Script10.ksh After successful completion of Script1.ksh I need to run Script7.ksh. After successful... (4 Replies)
Discussion started by: ford2020
4 Replies

6. UNIX for Dummies Questions & Answers

at jobs not running

im doing at now <return> touch ~/Desktop/file.txt <ctrl d> and file is never created though it says job 7 at Wed Mar 17 15:15:55 2010 why? (4 Replies)
Discussion started by: glev2005
4 Replies

7. Shell Programming and Scripting

jobs run parallel - server consumption?

I have a requirement where jobs/scripts need to be run in the background.The concern here is there are around 20 scripts which need to be run in the bg.Does running all the 20 scripts/job at the same time in bg consumes much sever-utilization. If so wot would be the efficient way to run the jobs... (5 Replies)
Discussion started by: michaelrozar17
5 Replies

8. Shell Programming and Scripting

waiting on jobs in bash, allowing limited parallel jobs at one time, and then for all to finish

Hello, I am running GNU bash, version 3.2.39(1)-release (x86_64-pc-linux-gnu). I have a specific question pertaining to waiting on jobs run in sub-shells, based on the max number of parallel processes I want to allow, and then wait... (1 Reply)
Discussion started by: srao
1 Replies

9. UNIX for Dummies Questions & Answers

Running parallel process

i am having 4 process,have to run parallel and not after one by one. sample1.sh sample2.sh sample3.sh sample4.sh Thanks in advance. i (11 Replies)
Discussion started by: sagar_1986
11 Replies

10. Shell Programming and Scripting

Running script in Parallel

Hi Folks I have a doubt. I have a script which is running with 2 input parameters ./GetDSLnkCount.sh <jobnumber> Parmfile.txt I need to run the script in parallel for different jobnumbers. The commands are ./GetDSLnkCount.sh jnhuc14500 Parmfile.txt ./GetDSLnkCount.sh jnhuc14501... (3 Replies)
Discussion started by: morbid_angel
3 Replies
NPM-RUN-SCRIPT(1)                                                                                                                NPM-RUN-SCRIPT(1)

NAME
npm-run-script - Run arbitrary package scripts SYNOPSIS
npm run-script <command> [--silent] [-- <args>...] alias: npm run DESCRIPTION
This runs an arbitrary command from a package's "scripts" object. If no "command" is provided, it will list the available scripts. run[-script] is used by the test, start, restart, and stop commands, but can be called directly, as well. When the scripts in the package are printed out, they're separated into lifecycle (test, start, restart) and directly-run scripts. As of ` https://blog.npmjs.org/post/98131109725/npm-2-0-0, you can use custom arguments when executing scripts. The special option -- is used by getopt https://goo.gl/KxMmtG to delimit the end of the options. npm will pass all the arguments after the -- directly to your script: npm run test -- --grep="pattern" The arguments will only be passed to the script specified after npm run and not to any pre or post script. The env script is a special built-in command that can be used to list environment variables that will be available to the script at run- time. If an "env" command is defined in your package, it will take precedence over the built-in. In addition to the shell's pre-existing PATH, npm run adds node_modules/.bin to the PATH provided to scripts. Any binaries provided by locally-installed dependencies can be used without the node_modules/.bin prefix. For example, if there is a devDependency on tap in your package, you should write: "scripts": {"test": "tap test/*.js"} instead of "scripts": {"test": "node_modules/.bin/tap test/*.js"} to run your tests. The actual shell your script is run within is platform dependent. By default, on Unix-like systems it is the /bin/sh command, on Windows it is the cmd.exe. The actual shell referred to by /bin/sh also depends on the system. As of ` https://github.com/npm/npm/releases/tag/v5.1.0 you can customize the shell with the script-shell configuration. Scripts are run from the root of the module, regardless of what your current working directory is when you call npm run. If you want your script to use different behavior based on what subdirectory you're in, you can use the INIT_CWD environment variable, which holds the full path you were in when you ran npm run. npm run sets the NODE environment variable to the node executable with which npm is executed. Also, if the --scripts-prepend-node-path is passed, the directory within which node resides is added to the PATH. If --scripts-prepend-node-path=auto is passed (which has been the default in npm v3), this is only performed when that node executable is not found in the PATH. If you try to run a script without having a node_modules directory and it fails, you will be given a warning to run npm install, just in case you've forgotten. You can use the --silent flag to prevent showing npm ERR! output on error. You can use the --if-present flag to avoid exiting with a non-zero exit code when the script is undefined. This lets you run potentially undefined scripts without breaking the execution chain. SEE ALSO
o npm help 7 scripts o npm help test o npm help start o npm help restart o npm help stop o npm help 7 config January 2019 NPM-RUN-SCRIPT(1)
All times are GMT -4. The time now is 07:05 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy