Running jobs in parallel


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Running jobs in parallel
# 8  
Old 04-06-2011
Well I think post #6 is pretty much what methyl suggested.
# 9  
Old 04-06-2011
Thanks Chubler for the code. I ran it a couple of times and I got the same issue. Hits a long running sql script and ignores the others.

This is the output I was getting:

NUM= 0
[1] 3278
NUM= 1
[2] 3282
NUM= 2
[3] 3286
NUM= 3
[4] 3290
NUM= 4
[5] 3294
NUM= 5
[6] 3298
NUM= 6
NUM= 6
NUM= 6
NUM= 6
NUM= 6
NUM= 6
..........

It gets stuck at 6 and doesn't proceed further. I check the sqlplus jobs and I find nothing is running:

ps -ef|grep "sqlplus -s"

oracle 4341 15634 0 20:08:09 pts/1 0:00 grep sqlplus -s
# 10  
Old 04-06-2011
@gctex
Please post the complete and unedited code (after blotting confidential information with X's) which produced the output in post #9.

Ps. I suspect that I first became a certified Oracle DBA before you were born. There is much popular usage of basic computer terms in general circulation. In the context of your posts it is impossible to determine whether you are referring to a program written in Oracle SQL or a unix Script. In my above request I use the word "code" to refer to either.

Last edited by methyl; 04-06-2011 at 09:44 PM..
# 11  
Old 04-06-2011
@gctex, From the output (I left the NUM=$NUM debugging line in by mistake), we can see it has fired off 6 jobs and the fact that NUM=6 is poping up means the jobs are still running.

The process ID's started were 3278, 3282, 3286, 3290, 3294 and 3299

I dont know why your ps -ef | grep "sqlplus -s" didn't show them. Next time check the individual process with something like ps -fp 3278
This User Gave Thanks to Chubler_XL For This Post:
# 12  
Old 04-06-2011
@chubler, is there a quick way to trap "ksh: cnt=: bad number" errors?

My code is working just fine, just a couple of places it throws this bad number error, I need to see what value is being assigned, if it is not a number.
# 13  
Old 04-06-2011
Would help if you could show the code your running. I'd assume you are trying to do something with a variable called cnt, to test if cnt has a valid numeric try something like:

Code:
[ $cnt -gt 0 -o $cnt -le 0 ] 2> /dev/null || echo "cnt has bad numeric value of: $cnt"

# 14  
Old 04-06-2011
@Chubler, you are right, I was trying to find the number of sql processes running and assigning that to cnt variable. Here's the code:

while read sql
do

stuff "sqlplus"

let cnt=$cnt+1

if (( $cnt == 6 )) ; then
while true
do
if (( $(ps -ef|grep "sqlplus -s"|grep -v grep|wc -l) >= 6 )); then
sleep 5
elif (( $(ps -ef|grep "sqlplus -s"|grep -v grep|wc -l) < 6 )); then
let cnt=$(ps -ef|grep "sqlplus -s"|grep -v grep|wc -l)
[ $cnt -gt 0 -o $cnt -le 0 ] 2> /dev/null | echo "cnt has bad numeric value of : $cnt"
break
fi
done
fi
done < sqlscript.lst

(In your code snippet, do we need 2 pipes? I removed one, it worked fine)

This was the output:

[1] 5970
[2] 5972
[3] 5974
[4] 5975
[5] 5978
[6] 5980
ksh: cnt=: bad number
cnt has bad numeric value of : 0
[7] 6000
[8] 6002
[9] 6004
[10] 6005
[11] 6007
[12] 6011
ksh: cnt=: bad number
cnt has bad numeric value of : 0

But our error trap should print $cnt only if $cnt -gt or -lt 0, right? Not sure why it is printing when $cnt = 0.

And it looks like the error appears when the next batch of 6 sqls are processed, starting with the second batch.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Running script in Parallel

Hi Folks I have a doubt. I have a script which is running with 2 input parameters ./GetDSLnkCount.sh <jobnumber> Parmfile.txt I need to run the script in parallel for different jobnumbers. The commands are ./GetDSLnkCount.sh jnhuc14500 Parmfile.txt ./GetDSLnkCount.sh jnhuc14501... (3 Replies)
Discussion started by: morbid_angel
3 Replies

2. UNIX for Dummies Questions & Answers

Running parallel process

i am having 4 process,have to run parallel and not after one by one. sample1.sh sample2.sh sample3.sh sample4.sh Thanks in advance. i (11 Replies)
Discussion started by: sagar_1986
11 Replies

3. Shell Programming and Scripting

waiting on jobs in bash, allowing limited parallel jobs at one time, and then for all to finish

Hello, I am running GNU bash, version 3.2.39(1)-release (x86_64-pc-linux-gnu). I have a specific question pertaining to waiting on jobs run in sub-shells, based on the max number of parallel processes I want to allow, and then wait... (1 Reply)
Discussion started by: srao
1 Replies

4. Shell Programming and Scripting

jobs run parallel - server consumption?

I have a requirement where jobs/scripts need to be run in the background.The concern here is there are around 20 scripts which need to be run in the bg.Does running all the 20 scripts/job at the same time in bg consumes much sever-utilization. If so wot would be the efficient way to run the jobs... (5 Replies)
Discussion started by: michaelrozar17
5 Replies

5. UNIX for Dummies Questions & Answers

at jobs not running

im doing at now <return> touch ~/Desktop/file.txt <ctrl d> and file is never created though it says job 7 at Wed Mar 17 15:15:55 2010 why? (4 Replies)
Discussion started by: glev2005
4 Replies

6. Shell Programming and Scripting

Conditional execution and parallel jobs

how can i process jobs parallel with conditions below. Script1.ksh Script2.ksh Script3.ksh Script4.ksh Script5.ksh Script6.ksh Script7.ksh Script8.ksh Script9.ksh Script10.ksh After successful completion of Script1.ksh I need to run Script7.ksh. After successful... (4 Replies)
Discussion started by: ford2020
4 Replies

7. IP Networking

running servers parallel

I'm going to undertake a hardware refresh soon and I was wondering if it is possible to run two machines (X and Y) with the same hostname (but different IP addresses) on the same network? Server X is the original server and has an entry in DNS. Server Y is the new server and won't have an entry... (1 Reply)
Discussion started by: soliberus
1 Replies

8. Shell Programming and Scripting

Running scripts in parallel

Hi, Iam having the scripts as follows. i jus want to run those in parallel. main aim is to minimise the time for overall execution of the script. now out.txt is having 1 lac records. script1(split.sh) split -1000 out.txt splitout ls -A splitout* > filelist.txt cat filelist.txt... (6 Replies)
Discussion started by: nivas
6 Replies

9. Shell Programming and Scripting

Diff: Server n parallel jobs

Hi, Could any one please explain the difference between DataStage server edition jobs and DS parallel extender jobs...? In which scenarios or application areas do we use either of these jobs.? Regards Suresh (0 Replies)
Discussion started by: sureshg_sampat
0 Replies

10. Programming

running a parallel program

hi , i need to run a parallel program . for example; program1 { array=" the second program should called here : program 2" the execution should continue } the 2nd program should recieve an array of information as argument and it should... (4 Replies)
Discussion started by: bankpro
4 Replies
Login or Register to Ask a Question