Today (Saturday) We will make some minor tuning adjustments to MySQL.

You may experience 2 up to 10 seconds "glitch time" when we restart MySQL. We expect to make these adjustments around 1AM Eastern Daylight Saving Time (EDT) US.


How to detect and fix why crontab job is not executed?


Login or Register to Reply

 
Thread Tools Search this Thread
# 1  
How to detect and fix why crontab job is not executed?

I have set several cron jobs. I recently added a new cron job that copies a file of last day from another server and is executed each day (for example at 04:00 am) but when I check next day the file hasn't been copied.

I'm working in GNU/Linux CentOS (2.6.32) system.

The files that I need to copy are created in the origin server before 23:00 hours.

I had the cron job like below: (The first line is the job that fails. related script to copy the files is script1.sh)
Code:
	
[root@srvc ~]# crontab -l
0 6 * * * /path/to/scripts/script1.sh
0 5 * * * /path/to/scripts/script3.sh
0 8 * * 1 /path/to/scripts/script4.sh

When I checked next day about 11:00 am I see the file wasn't copied, I edited cron job to be executed at 11:10 am and the file was copied successfully.

Code:
    
[root@srvc ~]# crontab -l
10 11 * * * /path/to/scripts/script1.sh

The script1.sh content is this:
Code:
#!/bin/bash
 
dyear=`date +'%Y' -d "1 day ago"`
dmonth=`date +'%b' -d "1 day ago"`
ddate=`date +%Y-%m-%d -d "1 day ago"`

sshpass -p 'ThePassword' scp -r root@X.X.X.X:/path/to/files/*$ddate* /Destination/path/$dyear/$dmonth/

The files to be copied have in their name the format Logfile.2018-01-17

How to know why cron job fails and how to fix it?

Thanks in advance.`
# 2  
/path/to/files/*$ddate* is expanded on the calling host i.e. the destination host.
Quote the * characters to expand on the source host.
/path/to/files/\*$ddate\* or
/path/to/files/"*$ddate*" or
"/path/to/files/*$ddate*"
Note that within the " (double quotes) the $ddate is still expanded on the calling host.
# 3  
Is that behaviour reproducible? What's the difference between the script executing at 6:00h and 11:10h? Anything in the log files? If not, modify the script to log its steps. Is it possible the files does not (yet) exist on the source node at 6:00h?
# 4  
Quote:
Originally Posted by RudiC
modify the script to log its steps.
This was my first thought too. How about doing the following to investigate:

change the script like this:
Code:
#!/bin/bash
set -xv 
dyear=`date +'%Y' -d "1 day ago"`
dmonth=`date +'%b' -d "1 day ago"`
ddate=`date +%Y-%m-%d -d "1 day ago"`

sshpass -p 'ThePassword' scp -r root@X.X.X.X:/path/to/files/*$ddate* /Destination/path/$dyear/$dmonth/

Then modify your crontab like this:

Code:
10 11 * * * /path/to/scripts/script1.sh > /path/to/cronlog.log 2> /path/to/cronlog.err

and have a look at what is logged. My first suspect would be the unquoted globs too, like MadeInGermany already said.

Two things to notice: if you create cron jobs you should ALWAYS redirect their stdout and their stderr - either to a (log-)file or to /dev/null if you are not interested. Otherwise any output the script eventually generates creates a mail to root which you probably want to avoid.

Second, you should really, really do away with sshpass. Even the developers admit that it is ill advised to use it and it is offered just as a last straw effort. When such a process ist started you can see the password in cleartext in the process list - not to mention the script file itself. You might secure the script file against being read by everybody but the output of ps is public information.

You can easily try it yourself: open two terminal windows as a normal user to some host. Issue in one of them:

Code:
sshpass -p 'ThePassword' ssh root@X.X.X.X sleep 1000

Now issue in the other window, while this runs

Code:
ps -fe | grep [s]sh

and you will see the password there.

I hope this helps.

bakunin
# 5  
Quote:
Originally Posted by bakunin
...
My first suspect would be the unquoted globs too, like MadeInGermany already said.
...
Why would it fail at 6:00h but work at 11:10h, then?
# 6  
Hi to all,

Thanks for your answers.

IMHO the issue is not the expansion of the variable because it worked at 11:10 but not at 06:00 and the script1.sh is the same.

The files to be copied are created before 23:00 hours, so that wouldn't be the reason either.

In order to redirect the log like bakunin saiys, how to know which is the path for cronlog.log and cronlog.err?
# 7  
Quote:
Originally Posted by Ophiuchus
... which is the path for cronlog.log and cronlog.err?
This is up to you to chose. For temporary, transient debugging you might select your own home directory, for permanent logging, /var/log lends itself to usage.
Login or Register to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Detect changes to crontab

Dear All, My server is running crontabs of 4 different users. I want to develop a script that whenever a particular change occurs in a crontab , it is detected and the particular change is noted into a file. Kindly let me know of suggestions on how it can be achieved. My algo would be: ... (1 Reply)
Discussion started by: Junaid Subhani
1 Replies

2. UNIX for Dummies Questions & Answers

Cron job executed at wrong time

Dear *nix users. I'm on Mac OS 10.6 / Terminal and try to use crontab to schedule two scripts every 30 minutes and every 41 minutes. I followed the man instructions and created / installed a crontab file for the current user:crontab -e with the following content */30 * * * *... (4 Replies)
Discussion started by: moxnos
4 Replies

3. Shell Programming and Scripting

crontab job not executed with variables

Hi, I am trying to execute a script (for once) during the booting time in Ubuntu system. However, the result is only showing the strings without without the variables. Here is the script: MgrIp=$(ec2-describe-instances --filter tag:Name=Mgr --filter instance-state-name=running | egrep... (4 Replies)
Discussion started by: turki_00
4 Replies

4. Shell Programming and Scripting

Users who desire to have their .profile executed must explicitly do so in the crontab entry. Why?

The .profile file should be read when the user logs in. So, there should be no need to execute .profile file again in a cron job (since the cron job is run after the user logs in). Doesn't the cron require login from the user. Then, from where does the cron execute? Please help!! (1 Reply)
Discussion started by: thulasidharan2k
1 Replies

5. UNIX and Linux Applications

Log files to view job executed in Autosys -- Help ASAP

Hi all, I really need your help ASAP on this. Below is the description of my problem and a sketch of Autosys Job Activity Console ++++++++++++++++++++++++++++++++ File View Options +++++++++++++++++++++++++++++++ Job Name Description Status Command Machine... (1 Reply)
Discussion started by: sakal_woman
1 Replies

6. Shell Programming and Scripting

Script errors out only when its executed via job

I wrote a script to shutdown the oracle database. The script works fine when I manually run the script. However, when i schedule a job, i get the following error. Shutting Down cmismart .................... ORA-01034: ORACLE not available ORA-27101: shared memory realm does not exist SVR4... (6 Replies)
Discussion started by: mrx1350
6 Replies

7. Shell Programming and Scripting

Routine doesn't give output when executed in crontab

I have a script running in the crontab that gets data from a database every hour. Now I would like to execute a fortran routine to process the data in some way, after getting it and saving it locally. I have added the following commands to my script: set convert =... (1 Reply)
Discussion started by: SharkM
1 Replies

8. Shell Programming and Scripting

bash scripting cannot executed in crontab

hi guys, i have a problem. a week ago i made a successful crontab that execute bash scripting daily, it worked well but now, it doesn't work at all, in the mail i have: " /home/jimmy/cha/scripts/cekpderr produced the following output: lagi jalan /home/jimmy/cha/scripts/cekpderr:... (6 Replies)
Discussion started by: jimmbp
6 Replies

9. Shell Programming and Scripting

Shell script doesn't get executed using crontab

I have the following crontab entry to run a shell script for every 30 minutes of every day: 30 * * * * $HOME/main.sh > $HOME/main.log 2>$HOME/error.log after I created the crontab file I have also done: $crontab my_crontab I also check to make sure it exists, by using the following... (11 Replies)
Discussion started by: radhika
11 Replies

10. Shell Programming and Scripting

ant not being executed as cron job

i have a script that uses an ant build.xml and its targets to pull a project from a cvs server, attempt to build the project, and then email me the results. When I run the script (either @ CLI or as a cron job) while I am logged in, everything works fine. However, if the script is set up to run... (5 Replies)
Discussion started by: kingfinny
5 Replies

Featured Tech Videos