Shell Programming and Scripting

Hi Experts,

I have a shell script called "updatevs" that is scheduled to run at 6.00 am everyday via cronjob. The cronjob will execute this script and output to a log file. The functionality of this script is to read the database and run a set of commands. This script is generally successful but some times due to network outage or Database being down it fails.

Only in case of a failure i would like to rerun the script every 20 mins for the next 2 hours, if still not successful after 2 hours i would like an email to be triggered to address "xxxxx@abc.com"

There may be several ways to solve this case however I was thinking of writing a small script called "checklog" which reads the log file and searching for the obevious words like "ERROR", when found "Checklog" script will rerun "updatevs" script. I don't think this is the solution as i can't see a way of making this recurring.

Please let me know your valuable feeback on how to acheive this.

Many Thanks

Rather than doing a search in the log files to check if the script updatevs completed successfully or not, I think a better way to do accomplish would be to do the following:
1. Modify the script updatevs to exit with a non-zero return code on failures.
2. Write another wrapper script that would call updatevs and based on the return code of the script decide whether it needs to be called again..

Just a sample of how the wrapper would look like (Not tested):

Code:

#New Job for updatevs
typeset -i exitStatus=1
typeset -i retryCntr=0
function callUpdateEnv
{
  . updatevs
  return $?
}

while [ $exitStatus != 0 ]
do
   
   callUpdateEnv
   exitStatus=$?
   if (( $exitStatus != 0 )); 
   then 
      (( retryCntr = ${retryCntr} + 1 ))
      if (( $retryCntr > 5 ));
         mailx -s "Job failed" xxxxx@abc.com < /dev/null
      fi
      sleep 1200
   fi
   
done

echo "completed successfully"

There would be several other better ways to do it but going through the logs doesn't seem to be the right one...

regards,
Arun.

Code:

n=0
while ! updatevs
do
  [ $n -le 6 ] && sleep 1200
  if [ $n -gt 6 ]
  then
     echo whatever | mail -s "updatevs failed" xxxxx@abc.com
     exit 1
  fi
  n=$(( $n + 1 ))
done

Hi Arun,

Many thanks for your solution.

This is what i have done to script updatevs to exit with a non-zero code on failures, please suggest if this is correct or not

###################
if [$? -ne "0"]
exit
###################

These lines are added after the last command executed in script "updatevs"

I created a new file called wrapper and have got your code in it but when i run wrapper is reports the following error

syntax error at line 19: 'fi' unexpected

Any suggestions appreciated.

Many thanks

In your script updatevs, I would say add exit $? at the end of the script. That way, the wrapper would be able to trap the correct return code.

The wrapper was failing because of a missing then

Code:

Instead of:
   if (( $retryCntr > 5 ));
Use:
   if (( $retryCntr > 5 )); then

regards,
Arun.

Arun,

Thanks again. The wrapper script seems to work with out any error messages however it does not produce any email. Here is what i did:

changed the sleep value to 30 in Wrapper so i need not have to wait long to see the eamil.
Ran wrapper.

wrapper seems to execute updatevs script successfully but i was expecting wrapper script to display the message "completed successfully".

commented sleep in wrapper, I modified "updatevs" script to ensure it fails hoping the wrapper to send an email but wrapper does not send any mail.

The mailx command does send out an email when this command is run on its own.

Please can you see where i am going wrong.

Can you please paste here the code for both the scripts that you are using? Would be much more easier for me to find what the issue is...

regards,
Arun.