Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to WAIT for jobs in each group to finish?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to WAIT for jobs in each group to finish?
# 1  
Old 09-10-2011
How to WAIT for jobs in each group to finish?
I have the shell script to call a Perl routine and pass the Informatica WorkFlow name to it. Jobs in each group executes in background do not seem to wait at all. How do I make it to WAIT for the prior group to complete before execute the next group of jobs?

Sample of the jobs flow:

Code:
{
 
 echo "=========>> Executing Group-1 at `date` "
 {
   `./ProcessData.pl wf_LOAD_EDW_Table1 >> ${LogDir}/ProcessDataGroup1.log 2>&1 &`;  
   `./ProcessData.pl wf_LOAD_EDW_Table2 >> ${LogDir}/ProcessDataGroup1.log 2>&1 &`;  
 }
 wait
 echo "=========>> Executing Group-1 Has Completed at `date` "
 echo ""
 echo "=========>> Executing Group-2 at `date` "
 {
   `./ProcessData.pl wf_LOAD_EDW_Table3 >> ${LogDir}/ProcessDataGroup2.log 2>&1 &`;  
   `./ProcessData.pl wf_LOAD_EDW_Table4 >> ${LogDir}/ProcessDataGroup2.log 2>&1 &`;  
 }
 wait
 echo "=========>> Executing Group-2 Has Completed at `date` "
 echo ""
 echo "=========>> Executing Group-3 at `date` "
 { 
   `./ProcessData.pl wf_LOAD_EDW_Table5 >> ${LogDir}/ProcessDataGroup3.log 2>&1 &`;  
   `./ProcessData.pl wf_LOAD_EDW_Table6 >> ${LogDir}/ProcessDataGroup2.log 2>&1 &`;  
 }
 wait
 echo "=========>> Executing Group-3 Has Completed at `date` "

}

Last edited by pludi; 09-10-2011 at 05:36 PM..
# 2  
Old 09-10-2011
Is there any reason for which you are putting your commands into backticks (`...`)? Try removing them and run the script again.
# 3  
Old 09-10-2011
Code:
echo "=========>> EDW_Daily_Sch_DS_step7 Group-1 Job Started at `date` "
   `sleep 10 &`
   `sleep 30 &`
 wait
 echo "=========>> EDW_Daily_Sch_DS_step7 Group-1 Has Completed at `date` "
 echo ""
 echo "=========>> EDW_Daily_Sch_DS_step7 Group-2 Job Started at `date` "
   `sleep 30 &`
   `sleep 10 &`
 wait
 echo "=========>> EDW_Daily_Sch_DS_step7 Group-2 Has Completed at `date` "

Result:
Code:
=========>> EDW_Daily_Sch_DS_step7 Group-1 Job Started at Sat Sep 10 13:47:38 PDT 2011
=========>> EDW_Daily_Sch_DS_step7 Group-1 Has Completed at Sat Sep 10 13:48:18 PDT 2011

=========>> EDW_Daily_Sch_DS_step7 Group-2 Job Started at Sat Sep 10 13:48:18 PDT 2011
=========>> EDW_Daily_Sch_DS_step7 Group-2 Has Completed at Sat Sep 10 13:48:58 PDT 2011

Should it wait for only 30 sec for each group?
Last edited by Franklin52; 09-12-2011 at 03:06 AM.. Reason: Please use code tags for code and data samples, thank you
# 4  
Old 09-10-2011
Let me ask you again why are you using backticks? What shell are you using? Try the code without backticks:
Code:
#!/bin/bash
echo "=========>> EDW_Daily_Sch_DS_step7 Group-1 Job Started at `date` "
sleep 10 &
sleep 30 &
wait
echo "=========>> EDW_Daily_Sch_DS_step7 Group-1 Has Completed at `date` "
echo ""
echo "=========>> EDW_Daily_Sch_DS_step7 Group-2 Job Started at `date` "
sleep 30 &
sleep 10 &
wait
echo "=========>> EDW_Daily_Sch_DS_step7 Group-2 Has Completed at `date` "

# 5  
Old 09-11-2011
It works now by move the & after the backtick. it needs it bcuz the command has more than 1 parm. the sleep ### is only the sample.
Code:
`CallWorkFlow wf_one > logs 2>&1` &
`CallWorkFlow wf_two > logs 2>&1` &
`CallWorkFlow wf_three > logs 2>&1` &
wait
`CallWorkFlow wf_four > logs 2>&1` &
`CallWorkFlow wf_five > logs 2>&1` &
`CallWorkFlow wf_six > logs 2>&1` &
wait

Last edited by Franklin52; 09-12-2011 at 03:06 AM.. Reason: Please use code tags for code and data samples, thank you
# 6  
Old 09-13-2011
What is the best way to check for return-code for each CallWorkFLow routine and abort the main job if error returns from any of the call?
# 7  
Old 09-13-2011
Quote:
Originally Posted by lv99
It works now by move the & after the backtick. it needs it bcuz the command has more than 1 parm.
For the third time, why is everything in backticks? Try removing all the pointless backticks.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script to run multiple jobs and it's dependent jobs

I have multiple jobs and each job dependent on other job. Each Job generates a log and If job completed successfully log file end's with JOB ENDED SUCCESSFULLY message and if it failed then it will end with JOB ENDED with FAILURE. I need an help how to start. Attaching the JOB dependency... (3 Replies)
Discussion started by: santoshkumarkal
3 Replies

2. UNIX for Dummies Questions & Answers

How to wait for a grand child to finish?

Hello all, I have a very basic question. I have a requirement where in, I have a main process which forks a child process, which execs and runs a c executable corresponding to a daemon. In the c executable corresponding to a daemon, as everyone does, I fork another child process, and as part of... (7 Replies)
Discussion started by: sai2krishna
7 Replies

3. Shell Programming and Scripting

calling a shell script in background and wait using "wait" in while loop

Hi, I am facing a strange issue, when i call a script from my while loop in background it doesnt go in background, despite the wait i put below the whil loop it goes forward even before the process put in background is completed. cat abc.txt | while read -u4 line do #if line contains #... (2 Replies)
Discussion started by: mihirvora16
2 Replies

4. Shell Programming and Scripting

How to wait for tar to finish before sending the archive through ftp?

Hey all, I want to automate tarring a directory then using ftp to transfer the files over. I was able to put the commands together but what I'm noticing is that only the very first file is being tarred and then transferred. tar cvpf new.backup sourceAbove is the command I'm using which works... (4 Replies)
Discussion started by: Keepcase
4 Replies

5. Shell Programming and Scripting

waiting on jobs in bash, allowing limited parallel jobs at one time, and then for all to finish

Hello, I am running GNU bash, version 3.2.39(1)-release (x86_64-pc-linux-gnu). I have a specific question pertaining to waiting on jobs run in sub-shells, based on the max number of parallel processes I want to allow, and then wait... (1 Reply)
Discussion started by: srao
1 Replies

6. Shell Programming and Scripting

wait command - cat it wait for not-chile process?

Did not use 'wait' yet. How I understand by now the wait works only for child processes, started background. Is there any other way to watch completion of any, not related process (at least, a process, owned by the same user?) I need to start a background process, witch will be waiting... (2 Replies)
Discussion started by: alex_5161
2 Replies

7. Shell Programming and Scripting

How to wait for the subprocess to finish in tcl

Hi All Here i have a piece of code, set filename "./GopiRun.sh" #I need to wait here until the GopiRun.sh is completed how do i achive this exit. (1 Reply)
Discussion started by: nathgopi214
1 Replies

8. Shell Programming and Scripting

background jobs exit status and limit the number of jobs to run

i need to execute 5 jobs at a time in background and need to get the exit status of all the jobs i wrote small script below , i'm not sure this is right way to do it.any ideas please help. $cat run_job.ksh #!/usr/bin/ksh #################################### typeset -u SCHEMA_NAME=$1 ... (1 Reply)
Discussion started by: GrepMe
1 Replies

9. Shell Programming and Scripting

Need to execute 2 scripts, wait, execute 2 more wait, till end of file

:cool: I need to execute a shell script to do the following: cat a file run two back ground processes using the first two values from the file wait till those background processes finish run two more background processes using the next two values from the file wait till those background... (1 Reply)
Discussion started by: halo98
1 Replies

10. UNIX for Advanced & Expert Users

How can I wait for PID to finish in another shell

Have a need to schedule programs that can run after other programs are completed. Here's the catch: 1) The list of programs will not always be the same (kind of a plug-n-play deal) 2) The invoking shell may not be the same as the shell of the program being waited on In other words, I need... (2 Replies)
Discussion started by: superdelic
2 Replies
Login or Register to Ask a Question