How to detect and fix why crontab job is not executed?


Login or Register to Reply

 
Thread Tools Search this Thread
# 1  
Old 2 Weeks Ago
How to detect and fix why crontab job is not executed?

I have set several cron jobs. I recently added a new cron job that copies a file of last day from another server and is executed each day (for example at 04:00 am) but when I check next day the file hasn't been copied.

I'm working in GNU/Linux CentOS (2.6.32) system.

The files that I need to copy are created in the origin server before 23:00 hours.

I had the cron job like below: (The first line is the job that fails. related script to copy the files is script1.sh)
Code:
	
[root@srvc ~]# crontab -l
0 6 * * * /path/to/scripts/script1.sh
0 5 * * * /path/to/scripts/script3.sh
0 8 * * 1 /path/to/scripts/script4.sh

When I checked next day about 11:00 am I see the file wasn't copied, I edited cron job to be executed at 11:10 am and the file was copied successfully.

Code:
    
[root@srvc ~]# crontab -l
10 11 * * * /path/to/scripts/script1.sh

The script1.sh content is this:
Code:
#!/bin/bash
 
dyear=`date +'%Y' -d "1 day ago"`
dmonth=`date +'%b' -d "1 day ago"`
ddate=`date +%Y-%m-%d -d "1 day ago"`

sshpass -p 'ThePassword' scp -r root@X.X.X.X:/path/to/files/*$ddate* /Destination/path/$dyear/$dmonth/

The files to be copied have in their name the format Logfile.2018-01-17

How to know why cron job fails and how to fix it?

Thanks in advance.`
# 2  
Old 2 Weeks Ago
/path/to/files/*$ddate* is expanded on the calling host i.e. the destination host.
Quote the * characters to expand on the source host.
/path/to/files/\*$ddate\* or
/path/to/files/"*$ddate*" or
"/path/to/files/*$ddate*"
Note that within the " (double quotes) the $ddate is still expanded on the calling host.
# 3  
Old 2 Weeks Ago
Is that behaviour reproducible? What's the difference between the script executing at 6:00h and 11:10h? Anything in the log files? If not, modify the script to log its steps. Is it possible the files does not (yet) exist on the source node at 6:00h?
# 4  
Old 2 Weeks Ago
Quote:
Originally Posted by RudiC
modify the script to log its steps.
This was my first thought too. How about doing the following to investigate:

change the script like this:
Code:
#!/bin/bash
set -xv 
dyear=`date +'%Y' -d "1 day ago"`
dmonth=`date +'%b' -d "1 day ago"`
ddate=`date +%Y-%m-%d -d "1 day ago"`

sshpass -p 'ThePassword' scp -r root@X.X.X.X:/path/to/files/*$ddate* /Destination/path/$dyear/$dmonth/

Then modify your crontab like this:

Code:
10 11 * * * /path/to/scripts/script1.sh > /path/to/cronlog.log 2> /path/to/cronlog.err

and have a look at what is logged. My first suspect would be the unquoted globs too, like MadeInGermany already said.

Two things to notice: if you create cron jobs you should ALWAYS redirect their stdout and their stderr - either to a (log-)file or to /dev/null if you are not interested. Otherwise any output the script eventually generates creates a mail to root which you probably want to avoid.

Second, you should really, really do away with sshpass. Even the developers admit that it is ill advised to use it and it is offered just as a last straw effort. When such a process ist started you can see the password in cleartext in the process list - not to mention the script file itself. You might secure the script file against being read by everybody but the output of ps is public information.

You can easily try it yourself: open two terminal windows as a normal user to some host. Issue in one of them:

Code:
sshpass -p 'ThePassword' ssh root@X.X.X.X sleep 1000

Now issue in the other window, while this runs

Code:
ps -fe | grep [s]sh

and you will see the password there.

I hope this helps.

bakunin
# 5  
Old 2 Weeks Ago
Quote:
Originally Posted by bakunin
...
My first suspect would be the unquoted globs too, like MadeInGermany already said.
...
Why would it fail at 6:00h but work at 11:10h, then?
# 6  
Old 2 Weeks Ago
Hi to all,

Thanks for your answers.

IMHO the issue is not the expansion of the variable because it worked at 11:10 but not at 06:00 and the script1.sh is the same.

The files to be copied are created before 23:00 hours, so that wouldn't be the reason either.

In order to redirect the log like bakunin saiys, how to know which is the path for cronlog.log and cronlog.err?
# 7  
Old 2 Weeks Ago
Quote:
Originally Posted by Ophiuchus
... which is the path for cronlog.log and cronlog.err?
This is up to you to chose. For temporary, transient debugging you might select your own home directory, for permanent logging, /var/log lends itself to usage.
Login or Register to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

More UNIX and Linux Forum Topics You Might Find Helpful
Crontab job is not working ipfreak UNIX for Dummies Questions & Answers 6 03-06-2016 04:39 AM
Detect changes to crontab Junaid Subhani Shell Programming and Scripting 1 08-28-2013 05:15 AM
Crontab Job Syntax JolietJake Shell Programming and Scripting 1 06-28-2013 09:43 PM
Cron job executed at wrong time moxnos UNIX for Dummies Questions & Answers 4 12-07-2012 12:45 PM
crontab job not executed with variables turki_00 Shell Programming and Scripting 4 10-04-2012 09:01 AM
Users who desire to have their .profile executed must explicitly do so in the crontab entry. Why? thulasidharan2k Shell Programming and Scripting 1 07-29-2011 02:36 AM
Schedule a job using Crontab jatanig Shell Programming and Scripting 1 03-15-2010 09:45 AM
Log files to view job executed in Autosys -- Help ASAP sakal_woman UNIX and Linux Applications 1 05-22-2009 07:47 PM
Script errors out only when its executed via job mrx1350 Shell Programming and Scripting 6 03-26-2009 01:50 PM
Routine doesn't give output when executed in crontab SharkM Shell Programming and Scripting 1 02-12-2008 09:56 AM
Help with a crontab job!!! kumarsaravana_s Shell Programming and Scripting 4 03-28-2007 04:02 AM
bash scripting cannot executed in crontab jimmbp Shell Programming and Scripting 6 11-11-2005 10:23 AM
crontab job Moe UNIX for Dummies Questions & Answers 3 11-02-2005 04:17 PM
Shell script doesn't get executed using crontab radhika Shell Programming and Scripting 11 06-09-2005 04:28 PM
ant not being executed as cron job kingfinny Shell Programming and Scripting 5 11-12-2004 12:11 PM