Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Run script in parallel in while loop

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Run script in parallel in while loop
# 1  
Old 09-22-2014
Run script in parallel in while loop
Hi
I am running a loop which actually runs same script for different argument value passed to it.
Code:
while read repID
do
echo "Starting for $repID";
date;
perl process_report.pl $repID 
done<${FILE_TO_READ}

However this runs in sequence.
I want the loop to not to wait for perl to complete before moving to next record.
So basically , it should call the perl script and move to next line of while loop. I want to run multiple instances of perl via while loop.
Is adding "&" will work fine ?

Thanks in advance
--Raj
# 2  
Old 09-22-2014
Hi dashing201

Yes this (&) should work.
Also, you could embrace it in brackets (perl process_report.pl $repID) .

This behaves the same (in this situation, there's a diffrence between the two i cannot explain), as long you dont want to fetch variables, but from the read, i'd guess you're writing the reports of that specific id...

Hope this helps
# 3  
Old 09-22-2014
Hello,

You can run the scripts in background, like following is an example for sleep process run in background.

Lets say a file in which number of mins/seconds have been given.(Input file).
Code:
cat file23
60
50
80
20
10
100

Now lets say we have a shell script same like your perl script. which needs to be executed in while loop.

Code:
cat test2.ksh
Q=$1
echo "starting sleep for" $Q "mins."
sleep $Q

Here is the example of While loop for same.

Code:
while read line
do
echo "Executing for" $line
./test2.ksh $line &
done < "file23"
Executing for 60
[1] 20921
Executing for 50
[2] 20922
Executing for 80
[3] 20923
Executing for 20
starting sleep for 60 mins.
[4] 20924
Executing for 10
starting sleep for 50 mins.
[5] 20931
starting sleep for 80 mins.
Executing for 100
[6] 20934

Hope following example may help you, similarly you can too put all proccesses in background while calling your perl script use at last
perl process_report.pl $repID &.

Thanks,
R. Singh
Last edited by RavinderSingh13; 09-22-2014 at 09:06 AM..
# 4  
Old 09-22-2014
Hi Ravinder/Sea

Thanks for the quick reply.
However if I have to implement a check to confirm if all the process have been completed then how can I do it ?
It is required to make sure that all parallel instances are completed before I change status to complete.

Thanks
Raj
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Linux

Inconsistency with parallel run

Hi All, I am running a parallel processing on aggregating a file. I am splitting the process into 7 separate parallel process and processing the same input file and the process will do the same for each 7 run. The issue I am having is for some reason the 1st parallel processes complete first... (7 Replies)
Discussion started by: arunkumar_mca
7 Replies

2. Shell Programming and Scripting

Run rest of script after parallel steps have completed

Hello gurus, I produce a number of .loc files in parallel depending on number of .csv in the folder for file in *csv do ./process.sh $file > $file.loc & done then I want to compile all the output from the previous step into a single masterlocfile and then perform the rest of the steps... (2 Replies)
Discussion started by: senhia83
2 Replies

3. Shell Programming and Scripting

Run the for loop in parallel

I have the below code which runs on multiple databases , but this runs one-after-one. I will need this to run in parallel so that i could save a lot of time. Please help!!! Thanks in advance for Db in `cat /var/opt/oracle/oratab |egrep -v "ASM" |grep -v \# |cut -d\: -f1` do { export... (5 Replies)
Discussion started by: jjoy
5 Replies

4. Shell Programming and Scripting

Script for telnet and run one command kill it and run another command using while loop

( sleep 3 echo ${LOGIN} sleep 2 echo ${PSWD} sleep 2 while read line do echo "$line" PID=$? sleep 2 kill -9 $PID done < temp sleep 5 echo "exit" ) | telnet ${HOST} while is executing only command and exits. (5 Replies)
Discussion started by: sooda
5 Replies

5. Shell Programming and Scripting

Run a script in parallel

Hey, I am new to UNIX scripting . I have script (ex: start_script) that starts a job in 10 different servers one server after another.Now I want to modify the script so that the script starts the job in all servers parallely (at a time in all servers).and I need the choice of selecting the... (3 Replies)
Discussion started by: mpspsm
3 Replies

6. Shell Programming and Scripting

script - how to prevent in parallel run

I have one shell script which is being accessed by many jobs at same time. I want to make the script such that , other job should wait for the script if script is being used by some other job. Is there any way to implement it in script level ? Gops (1 Reply)
Discussion started by: Gopal_Engg
1 Replies

7. Shell Programming and Scripting

Need to run same script multiple times in parallel

Hi all, I have a requirement in which a script invokes a Java program. Lets say script ABC invokes a java program with cfg file a parameter. This script takes 10 minutes to execute . Like this ineed to run the program 10 times meaning 100 minutes if i do it sequentially. If i open... (2 Replies)
Discussion started by: rahman_riyaz
2 Replies

8. Shell Programming and Scripting

Script to run infinite loop

Hi all, I have a script which triggers batch admin manager and gets the top 10 jobs and their status info. the output of this script is the list of all these jobs. I want to run this in infinite loop which will show top 100 jobs' status. the script is as follows #!/bin/sh exec &> capture1.txt... (1 Reply)
Discussion started by: digitalrg
1 Replies

9. UNIX for Advanced & Expert Users

Run a script parallel for a month's worth:

Is there a utility that can be used in a shell script that would run a .sql file for 30 or 31 days in a month at the same time parallely. Please Advice. Thanks SD12. (2 Replies)
Discussion started by: sd12
2 Replies

10. Shell Programming and Scripting

Run a same script in parallel with diffs parameters

i have script say some_script.ksh that takes an argument I need to run some_script.ksh in background parallely at the sametime with different arguments. Once all the background jobs complete, i need to run this script again in parallel with another 5 set of arguments. Would really... (1 Reply)
Discussion started by: hyennah
1 Replies
Login or Register to Ask a Question