Running jobs in parallel


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Running jobs in parallel
# 15  
Old 04-07-2011
The below code is still returning zero when you have 6 processes running
Code:
ps -ef|grep "sqlplus -s"|grep -v grep|wc -l

You can see that the number of jobs goes to [6] in the output, you then get the error about cnt=0 and then in jumps to [12].

My code was doing the correct thing to start with. It started 6 jobs and was was waiting for one of them to finish. You did ps -ef | grep "sqlplus -s" and didn't see anything so you assumed no jobs were started. The issue was that the processes aren't being picked up by the grep search.

As I suggested before start a job and then do ps -fp <pid> to find out what the command line is, this will help debug why grep "sqlplus -s" is not matching it.


The test on numeric value should have || this is "or" in shell not "pipe".
# 16  
Old 04-07-2011
Oh ok, got it about || ... forgot about it ... Smilie

For me "ps -ef|grep "sqlplus -s"|grep -v grep|wc -l" is showing the number of sessions running correctly. It shows 6 processes or less when the script is running. Not sure why you are getting 0.

Your code with "jobs" and "wait" just didn't go beyond the first 6 sqls for me. This whole job involving 50 tables takes only 2 minutes but when running your code it processed only 6 sqls and just didn't move further even after 5 minutes. I had to kill (ctrl+C) it everytime. I just tried it again and it gives the same results, only 6 sqls are processed and 6 logfiles are generated.
# 17  
Old 04-07-2011
If you feel like debugging it just run jobs with output to the terminal before the sleep:

Code:
#!/bin/ksh
PROCS=6
SLEEP=20
while read LINE
do
     while true
     do
         NUM=$(jobs | wc -l)
         echo NUM=$NUM
         if [ $NUM -lt $PROCS ]
         then
             stuff "$LINE" &
             break
         else
             jobs
             echo "========"
             sleep $SLEEP
         fi
    done
done < textfile
# wait for ALL remaining processes
wait

It works fine if I run the sleep command as my background task, but I'm curious about what in your environment is causing jobs to still keep reporting stuff thats finished.
This User Gave Thanks to Chubler_XL For This Post:
# 18  
Old 04-07-2011
Yeah Chubler, something in our envirorment does behave weird. If I replace "ps -ef" with "jobs" it just doesn't go beyond the first 6 sqls. Anyways, "ps -ef" works fine for me, occasional bad number error is there, when the "ps -ef" count is 0.

Thanks a great lot Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Running script in Parallel

Hi Folks I have a doubt. I have a script which is running with 2 input parameters ./GetDSLnkCount.sh <jobnumber> Parmfile.txt I need to run the script in parallel for different jobnumbers. The commands are ./GetDSLnkCount.sh jnhuc14500 Parmfile.txt ./GetDSLnkCount.sh jnhuc14501... (3 Replies)
Discussion started by: morbid_angel
3 Replies

2. UNIX for Dummies Questions & Answers

Running parallel process

i am having 4 process,have to run parallel and not after one by one. sample1.sh sample2.sh sample3.sh sample4.sh Thanks in advance. i (11 Replies)
Discussion started by: sagar_1986
11 Replies

3. Shell Programming and Scripting

waiting on jobs in bash, allowing limited parallel jobs at one time, and then for all to finish

Hello, I am running GNU bash, version 3.2.39(1)-release (x86_64-pc-linux-gnu). I have a specific question pertaining to waiting on jobs run in sub-shells, based on the max number of parallel processes I want to allow, and then wait... (1 Reply)
Discussion started by: srao
1 Replies

4. Shell Programming and Scripting

jobs run parallel - server consumption?

I have a requirement where jobs/scripts need to be run in the background.The concern here is there are around 20 scripts which need to be run in the bg.Does running all the 20 scripts/job at the same time in bg consumes much sever-utilization. If so wot would be the efficient way to run the jobs... (5 Replies)
Discussion started by: michaelrozar17
5 Replies

5. UNIX for Dummies Questions & Answers

at jobs not running

im doing at now <return> touch ~/Desktop/file.txt <ctrl d> and file is never created though it says job 7 at Wed Mar 17 15:15:55 2010 why? (4 Replies)
Discussion started by: glev2005
4 Replies

6. Shell Programming and Scripting

Conditional execution and parallel jobs

how can i process jobs parallel with conditions below. Script1.ksh Script2.ksh Script3.ksh Script4.ksh Script5.ksh Script6.ksh Script7.ksh Script8.ksh Script9.ksh Script10.ksh After successful completion of Script1.ksh I need to run Script7.ksh. After successful... (4 Replies)
Discussion started by: ford2020
4 Replies

7. IP Networking

running servers parallel

I'm going to undertake a hardware refresh soon and I was wondering if it is possible to run two machines (X and Y) with the same hostname (but different IP addresses) on the same network? Server X is the original server and has an entry in DNS. Server Y is the new server and won't have an entry... (1 Reply)
Discussion started by: soliberus
1 Replies

8. Shell Programming and Scripting

Running scripts in parallel

Hi, Iam having the scripts as follows. i jus want to run those in parallel. main aim is to minimise the time for overall execution of the script. now out.txt is having 1 lac records. script1(split.sh) split -1000 out.txt splitout ls -A splitout* > filelist.txt cat filelist.txt... (6 Replies)
Discussion started by: nivas
6 Replies

9. Shell Programming and Scripting

Diff: Server n parallel jobs

Hi, Could any one please explain the difference between DataStage server edition jobs and DS parallel extender jobs...? In which scenarios or application areas do we use either of these jobs.? Regards Suresh (0 Replies)
Discussion started by: sureshg_sampat
0 Replies

10. Programming

running a parallel program

hi , i need to run a parallel program . for example; program1 { array=" the second program should called here : program 2" the execution should continue } the 2nd program should recieve an array of information as argument and it should... (4 Replies)
Discussion started by: bankpro
4 Replies
Login or Register to Ask a Question