[Solved] Running scripts in parallel


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting [Solved] Running scripts in parallel
# 1  
Old 11-28-2012
[Solved] Running scripts in parallel

i have script A and script B, both scripts have to run in parallel, my requirement is script A create table temp1, post creating it will run fr 4 hrs , script B has to start 0nly after creation of table temp1 ( which is done by script A) , again script B will run for 5 hrs

if i run sequencially it will run for 9 hrs, how can i run in parallel( but above condition should match) ..........
# 2  
Old 11-28-2012
You could have different scripts for the two tasks and run one (or both) in background. You will still have to use some sort of inter-process-communication to make sure the one starts only when the second is past a certain point of execution. The following (pseudo-code-) sketch would layout this:

Code:
Master-script:

rm /some/file     # make sure there is no leftover from previous runs
first_script &
while [ ! -e /some/file ] ; do
      sleep some time
done
second_script


first_script:
long_command       # commands to be executed before second_script starts
touch /some/file
other_commands     # commands to be executed after second_script is started

This would first start "first_script" and then wait until "/some/file" is created, upon which it starts "second_script". "first_script" would signal being past a certain point by creating the file "/some/file", upon which "second_script" would be started by "master".

Another option would be to use Korn shell ("ksh") and make use of its "coprocess facility". I suggest you read the man page for "ksh" on how to use it.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
# 3  
Old 11-28-2012
What I am about to say assumes that table temp1 is a file.

If it is then:

1. At the end of script B ensure that temp1 is either deleted or renamed so that it doesn't exist anymore.

2. At the start of script B, "test" (see man pages) to see if the file temp1 exists and if it doesn't then "sleep" (see man pages) for, say, 600 seconds to sleep the script. Only when temp1 exists will script B continue processing.

The above also assumes that temp1 is created fairly instantly so will either exist or not exist. If temp1 is created slowly, then get script A "touch" (see man pages) a file to use as a flag to script B which can then "test" for the existance of the flag file instead. Script B should delete the flag file before it finishes.

Hope that helps (and I have succeeded in making that clear enough).

Post any further questions.

Last edited by hicksd8; 12-01-2012 at 11:35 AM..
This User Gave Thanks to hicksd8 For This Post:
# 4  
Old 11-28-2012
One option is to move table creation step in script A to a new script then make both script A and B dependent on this new script and fire off in parallel.

Or, you could update script B to check for the existence of the table and if it does not exist yet have it sleep for some time and only continue once the table has been created.
This User Gave Thanks to mjf For This Post:
# 5  
Old 11-28-2012
Just invoke script_B right after creation of table: temp1 inside script_A:-

script_A
Code:
#!/bin/ksh

# Some code here

sqlplus -s ${user}@${pass}@${inst} << EOF
create table temp1 as select ... ;
exit
EOF

script_B & 

# Some code here

This User Gave Thanks to Yoda For This Post:
# 6  
Old 11-29-2012
Code:
 
scriptA.sh
touch letsgotempfile_$(date +'%Y%m%d%')

Code:
 
$ more scriptB.sh
#!/usr/bin/ksh
log_path='/data/mig04/logs'
timestamp=$(date +'%Y%m%d%')
while [ ! -f  /letsgotempfile_$(date +'%Y%m%d%') ]
do
sleep  5
echo " sleeping for 5 sec .............."
done
echo " Going to start B ........."
exit 0

above code is working !!!!!................

i can not invoke scriptB in script A because for both scripts i am passing diffrent parameters

---------- Post updated at 09:46 PM ---------- Previous update was at 04:04 PM ----------

thanks everyone for ur valuable inputs ........script is working fine Smilie

Moderator's Comments:
Mod Comment edit by bakunin: glad to hear that and thank you for posting your solution. I changed the threads title.

Last edited by bakunin; 11-29-2012 at 12:42 PM..
# 7  
Old 11-30-2012
My only comment would be to ask why you sleep only for 5 seconds?

You indicated previously that both scripts take several hours to run. Having one waking up and burning CPU cycles every 5 seconds might just be wasteful of CPU. Why don't you sleep for 10 minutes each time.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parallel bash scripts

Need some help to replace bash script with parallel to speed up job on multiple files (400files.list is the file contains the absolute path to those 400 files). The bash script is to run the same program over the files repetitively. My bash_script.sh is: for sample in `cat 400files.list`; do... (3 Replies)
Discussion started by: yifangt
3 Replies

2. Shell Programming and Scripting

Running script in Parallel

Hi Folks I have a doubt. I have a script which is running with 2 input parameters ./GetDSLnkCount.sh <jobnumber> Parmfile.txt I need to run the script in parallel for different jobnumbers. The commands are ./GetDSLnkCount.sh jnhuc14500 Parmfile.txt ./GetDSLnkCount.sh jnhuc14501... (3 Replies)
Discussion started by: morbid_angel
3 Replies

3. Shell Programming and Scripting

[Solved] Running scripts in parallel that issue prompt

Hi all - I am totally stuck here :wall I have been asked to write a shell script that does a few little things and then reads from a config file and kicks off an instance of another script, say scriptB.ksh for each line in the config file. These should all be run in parallel. This is all fine but... (2 Replies)
Discussion started by: sjmolloy
2 Replies

4. UNIX for Dummies Questions & Answers

Running parallel process

i am having 4 process,have to run parallel and not after one by one. sample1.sh sample2.sh sample3.sh sample4.sh Thanks in advance. i (11 Replies)
Discussion started by: sagar_1986
11 Replies

5. Shell Programming and Scripting

Execute scripts in Parallel

Hi I want to execute few scripts in Parallel. There is a Master Script (MS.ksh) which will call internally all the scripts we need to run in Parallel. Say there are three set of scripts : ABC_1.ksh --> ABC_2.ksh --> ABC_3.ksh (execute ABC_2 when ABC_1 is successful ; Execute ABC_3 only when... (6 Replies)
Discussion started by: dashing201
6 Replies

6. Shell Programming and Scripting

Running jobs in parallel

I need to process 50 sqlplus scripts which are listed in a text file. I need to develop a shell script that'll read this file and run these sqlplus scripts. At any point of time, the number of sqlplus scripts running shouldn't exceed 6. If any of the sqlplus scripts completes successfully then... (17 Replies)
Discussion started by: gctex
17 Replies

7. Shell Programming and Scripting

Executing scripts in Parallel

Hi All, I have 3 shell scripts, Script1,Script2 and Script3. Now I want to run Script1 and Script2 in parallel and Script3 should depend on successful completion of both Script1 and Script2. Could you please suggest an approach of acheiving this... Thanks in advance (2 Replies)
Discussion started by: itsme_maverick
2 Replies

8. IP Networking

running servers parallel

I'm going to undertake a hardware refresh soon and I was wondering if it is possible to run two machines (X and Y) with the same hostname (but different IP addresses) on the same network? Server X is the original server and has an entry in DNS. Server Y is the new server and won't have an entry... (1 Reply)
Discussion started by: soliberus
1 Replies

9. Shell Programming and Scripting

Running scripts in parallel

Hi, Iam having the scripts as follows. i jus want to run those in parallel. main aim is to minimise the time for overall execution of the script. now out.txt is having 1 lac records. script1(split.sh) split -1000 out.txt splitout ls -A splitout* > filelist.txt cat filelist.txt... (6 Replies)
Discussion started by: nivas
6 Replies

10. Programming

running a parallel program

hi , i need to run a parallel program . for example; program1 { array=" the second program should called here : program 2" the execution should continue } the 2nd program should recieve an array of information as argument and it should... (4 Replies)
Discussion started by: bankpro
4 Replies
Login or Register to Ask a Question