Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Job Execution in Parallel and Maintain Dependency

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Job Execution in Parallel and Maintain Dependency
# 1  
Old 02-10-2009
Job Execution in Parallel and Maintain Dependency
All,

We need to run multiple jobs in parallel with the dependencies intact.

I was trying to use the same with &, but in vain. Is there is any other way to resolve this issue.



I need to run
Set1
A.sh
B.sh
C.sh

Set2
D.sh
E.sh

Set3
F.sh
G.sh

Jobs Parallely

When Set1 Completes I need to run the below Set4 irrespective of other sets

Set4
AA.sh
BB.sh

When set2 completes I need to run below Set 5 irrespective of other sets

Set 5
CC.sh

When set3 completed I need to run below Set 6 irrespective of other sets

Set6
DD.sh

Once Set 4, 5, 6 completes I need to run set7

Set7
final.sh


How can I make this parallel and set the dependency also?

any help is appreciated.

Regards
Anand M
# 2  
Old 02-10-2009
Can anyone help me in this regard?

Any help would be great.

Thanks
Anand
# 3  
Old 02-10-2009
Hi ,

bundle the sets(A,B....) to one one scripts and try...
For parallel running normally using "&" ..

for dependency wait command u can use..
Hope this will help


Regards,
# 4  
Old 02-10-2009
Hi,

Try something like this...( i have not tested this code)

#!/bin/sh

./Set1.sh &
process_id_set1=`ps -ef | grep ora_q001_DM2XDB | grep -v grep | awk '{print $2}'`
./Set2.sh &
process_id_set2=`ps -ef | grep ora_q001_DM2XDB | grep -v grep | awk '{print $2}'`
./Set3.sh &
process_id_set3=`ps -ef | grep ora_q001_DM2XDB | grep -v grep | awk '{print $2}'`

wait process_id_set1
echo "Set1 Completed -- now start Set4"
./Set4.sh &

wait process_id_set2
echo "Set2 Completed -- now start Set5"
./Set5.sh &



Regards,
# 5  
Old 02-10-2009
sorry replace ora_q001_DM2XDB with ur script name
# 6  
Old 02-11-2009
Thanks for the reply. I will try the same.

Rgds
Anand
# 7  
Old 02-11-2009
what does & mean?
is the script run in background.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to make parallel execution on folder ?

I have few very huge files ~ 2 Billion rows of 130 column(CDR data) in a folder, I have written shell script need to read on each file in a folder and will create a new files based on some logic. But problem is it's taking time to create a new file due to the size , So i dont want to corrupt... (6 Replies)
Discussion started by: rspwilliam
6 Replies

2. UNIX for Advanced & Expert Users

Parallel Execution of Command

Hi All, We have a table that has to store around 80-100 million records. The table is partitioned by a column called Market Code. There are 30 partitions each corresponding to one MRKT_CD. The source of this table is a join between 3-4 other tables. We are loading this table through SQLPLUS... (2 Replies)
Discussion started by: jerome_rajan
2 Replies

3. Programming

Parallel Execution of Programs

Since there've been a few requests for a method to execute commands on multiple CPUs (logical or physical), with various levels of shell-, make-, or Perl-based solutions, ranging from well-done to well-meant, and mostly specific to a certain problem, I've started to write a C-based solution... (4 Replies)
Discussion started by: pludi
4 Replies

4. Shell Programming and Scripting

Parallel execution of script not syncronize

I am haveing 2 scripts, 1st script calls 2ed script for each parameter.(parameters are kept in a different txt file) 1st script for x in `cat Export_Tables_List.txt` do sh Exp_Table.sh $x & done echo -e "1) following tables are successfully exported : \n" > temp cat... (1 Reply)
Discussion started by: sbmk_design
1 Replies

5. Shell Programming and Scripting

parallel execution of script/ synchro problem

Hi everybody, In a csh script, i need to run 4 time the same prog with different parameters. What i want is to run them in parallel. for this i use the command toto1.sh & toto2.sh & toto3.sh & toto4.sh For this I have no problem. In fact, I need to wait until all the programs are over to... (2 Replies)
Discussion started by: Moumou
2 Replies

6. Shell Programming and Scripting

Conditional execution and parallel jobs

how can i process jobs parallel with conditions below. Script1.ksh Script2.ksh Script3.ksh Script4.ksh Script5.ksh Script6.ksh Script7.ksh Script8.ksh Script9.ksh Script10.ksh After successful completion of Script1.ksh I need to run Script7.ksh. After successful... (4 Replies)
Discussion started by: ford2020
4 Replies

7. Shell Programming and Scripting

Parallel Job Execution

Hi All, At present I am using a UNIX Script which is running a set of JOBS. These JOBS are to be repeated for 20 times, means the same set of JOBS are repeated by just passing different arguments (From 1 to 20). Is there any way by which I can execute them parallel? At present its all... (4 Replies)
Discussion started by: Prashantckc
4 Replies

8. High Performance Computing

Parallel Execution on Multiple System

Hi All, I am working on a project where I need to execute set of arguments (around 500) on a Simulator. If I execute this on one linux (RedHat 8.0) machine it will approximately takes 2~3 days. Since I am having more linux machines am thinking of executing these on different machines in... (7 Replies)
Discussion started by: 123an
7 Replies

9. AIX

Make : parallel execution

Hi, Is there any way to run parallel jobs using make command? I am using non-GNU make utility on AIX 4.3. I want to run 2 jobs symultaneously using make utility. Thanks. Suman (0 Replies)
Discussion started by: suman_jakkula
0 Replies

10. Shell Programming and Scripting

Parallel Execution

Hello, I wish to run parallel process forked from one script. Currently I doing is submit them in background. For example: --------------------------------------------------------------- #!/usr/bin/ksh process1 & process2 & process3 & ..... ..... #here I check for completion of... (4 Replies)
Discussion started by: RishiPahuja
4 Replies
Login or Register to Ask a Question