Visit Our UNIX and Linux User Community


Monitoring Crontabs


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Monitoring Crontabs
# 1  
Old 08-23-2009
Monitoring Crontabs

Hi So, here is a million doller question. Being a Unix and Linux Admin we all use cront jobs to automate our tasks. But what if we need to monitor the cronjob itself. Oh!!! boy that can be a pain.

Few Techniques:

1. Redirect the output of the cronjob to a log file.Smilie
2. Using MAILTO option to get a e-mail notification for success or failure of a cronjob.Smilie

But in case of large set up it seems to be a no value ideaSmilie. let take an example of a set and than decide what should be the best option:Smilie

1. LAMP technology.
2. 75 VPS with LAMP.
3. ALL VPS has PHP scripts in cron to wget a PHP file from the local website.
4. Almost 10 Cronjobs running per VPS.
5. Frequency of cronjobs are 1 min. So, each Cronjob execute 24*60 = 1440 times. Smilie

Now both of the above idea would be ideal with this senario. Because going to 75 invidual VPSs and look for the log file will pain to the core.
MAILTO option seems to be promising but getting 1440*10*75 ,ail in a day will become an another mailbomb.

So, My question is how we can monitor these cronjob. is there any tools available which checks these cronjobs and redirect the result to a particular web page. Life would be easier if we have any sort this. One page all the information.

Secondly can we use this MAILTO option to create something like this for us. Like using MAILTO option send the mails to a specific e-mail address which belongs to some ticketing sstem or some other type of unit which generates a report on HTML page or PHP page.

Friends i am not sure how meaningful this will be for you guys but it is actually a pain for me these days. Smilie

Any sort of help or comment be will a great help.Smilie
# 2  
Old 08-23-2009
At this stage, it's still a relatively small environment so cron still sounds viable. But if you have a task running every minute, consider the implications if it starts hanging, you'd blow out cron's lmit on concurrent jobs in just under two hours and all cron jobs would stop.

Might I suggest just having a while - do - sleep60 loop instead? You can use cron to watchdog that script if required too Smilie

But if you are going to want to scale this up at any point, I'd recommend moving away from cron altogether and going to something like control-m or one of the other fine centralised automation tools and save yourself a costly migration later on.
# 3  
Old 08-24-2009
Quote:
Originally Posted by Smiling Dragon
At this stage, it's still a relatively small environment so cron still sounds viable. But if you have a task running every minute, consider the implications if it starts hanging, you'd blow out cron's lmit on concurrent jobs in just under two hours and all cron jobs would stop.

Might I suggest just having a while - do - sleep60 loop instead? You can use cron to watchdog that script if required too Smilie

But if you are going to want to scale this up at any point, I'd recommend moving away from cron altogether and going to something like control-m or one of the other fine centralised automation tools and save yourself a costly migration later on.
Thanks Dude for replying. Having a dedicated scheduler make sense but somehow i dont find it very helpful in my environment.
See these jobs are very small jobs which last i guess a max 2 - 3 mins. that's why these are scheduled on the localhost only. More over getting a dedicated server as a scheduler which run it from other end and tries to archive the data from the webserver will be a extra effort for network. I have look from the network side as well. Becoz they are running on localhost so, network is not bothered about these jobs.
Secondly i completely agree with you that what if they hung we might end up in real dead zone. But i have already tried to cure that by putting a script which works like a watch dog on these scripts and which prevents them to raise more than what they expected too.

So, in my situation having a tool like control-m will be help, i am not very confident about it.

But my real question was is there something we can do to integrate cronjob with something which is UI and from there we can keep and eye on jobs.

I heards something called "hudson" which keeps tracks of concurrent jobs like cron jobs. but i never used it. Any idea bout this software...or any other software in mind to resolve this issue, or any out of the box integration in mind which can be helpful in this situation.
# 4  
Old 08-24-2009
not sure if I understand the question...

we just make our scripts very good at error checking.
it sends an email if there is an error.
otherwise if just does exit 0, with no output.

as long as it works we don't care about logs and errors and such.
# 5  
Old 08-24-2009
Quote:
Originally Posted by robsonde
not sure if I understand the question...

we just make our scripts very good at error checking.
it sends an email if there is an error.
otherwise if just does exit 0, with no output.

as long as it works we don't care about logs and errors and such.
Oops, i am sorry if am not understandable. Question is how can you monitor the cronjob if you have almost 1000 of them, what you have mention is absolutely right. but these are PHP script which directly related to the application it self. Now there could be N number of reasons for the failure of the script.

let me give you an idea of the crontab entries:

12 14 * * * /etc/webmin/cron/tempdelete.pl
*/5 * * * * wget -q -O /dev/null http://localhost/admin/cron.php
0 2 * * * /etc/webmin/fsdump/backup.pl 47211236803641
* * * * * curl -s The UNIX and Linux Forums - Learn UNIX and Linux from Experts
* * * * * curl -s The UNIX and Linux Forums - Learn UNIX and Linux from Experts
* * * * * curl -s The UNIX and Linux Forums - Learn UNIX and Linux from Experts
* * * * * curl -s The UNIX and Linux Forums - Learn UNIX and Linux from Experts
* * * * * curl -s The UNIX and Linux Forums - Learn UNIX and Linux from Experts
* * * * * curl -s The UNIX and Linux Forums - Learn UNIX and Linux from Experts
* * * * * curl -s The UNIX and Linux Forums - Learn UNIX and Linux from Experts
15 1,3 * * * /srv/www/htdocs/batch/CMCAttendanceoldJob.sh >>/root/wget_output
10 2 * * 1 wget -q -O /dev/null The UNIX and Linux Forums - Learn UNIX and Linux from Experts


So, here you can see an example of the crontab entry. So, these PHP scripts are basicall page from teh web site which pull and puts the data inside and outside of DB. they not very log running running scripts.
So, my concern is as i have mentioned there could be N number of reasons that these scripts fails so, any idea, tool by which you can monitor these. That what all script ran today and what all didn't dude what reason.

Previous Thread | Next Thread
Test Your Knowledge in Computers #237
Difficulty: Easy
Paul Baran was the first to build a theoretical model for communication using packet switching.
True or False?

10 More Discussions You Might Find Interesting

1. HP-UX

HP -UX 11.31 monitoring

Hi, i have joined newly as a hp-ux admin... previously worked in linux environment can anyone tell me how to monitor memory and cpu utilization in hpux?? hp-ux is not as flexible as linux as i understand... please help me out by giving the commands needed and way of troubleshooting if the... (12 Replies)
Discussion started by: Prasad@hp-ux
12 Replies

2. Shell Programming and Scripting

Network Monitoring

My Office Hours between 10 A.M to 5 P.M .I am managing 16 client PCs which is remotely placed. I want to know the network status of every client PCs.I need an automatic trigger mail ,when the network connection is lost in any one of the Client PCs during office hours.I am a self study learner.I... (9 Replies)
Discussion started by: kannansoft1985
9 Replies

3. Shell Programming and Scripting

Multiple Crontabs

I have a crontab job in my sun solaris server and basically i have a unix script in 3 different parts 1 part will do an auto ftp from a server 2 part will have some grep and awk related commands and generate certain files 3 part the files that are generated above will be auto ftpied again... (1 Reply)
Discussion started by: aravindj80
1 Replies

4. UNIX for Dummies Questions & Answers

FTP monitoring

Hello there, I have the following situation : I have a ftp server (PureFTPd)on which users can connect and put their files. The thing is I need to synchronize the files uploaded by the users to another server (so that everybody can download (via http) from that server the resources uploaded by... (6 Replies)
Discussion started by: spiriad
6 Replies

5. Shell Programming and Scripting

crontabs entry not working

Hi, I have a crontab entry like the one shown below. My aim is to run the script from 06.10pm to 06.10am. But it seems not working. 10 18-06 * * * /usr/local/proj/cgi-bin/printmsg.pl 1>> /usr/local/proj/cgi-bin/printmsg.pl 2>> /usr/local/proj/cronlogs/printmsg.log Is... (2 Replies)
Discussion started by: DILEEP410
2 Replies

6. Solaris

Monitoring Script

Hi, I want to write a script that will monitor cpu,mem usage and disk usage for entire day and data will be redirected to file, (4 Replies)
Discussion started by: manoj.solaris
4 Replies

7. Red Hat

Monitoring tools

Hi, In HPUX there is a grate monitor tools named GLANCE, which give you information on the disks load, memory usage, cpu ... What is the equivalent tool in LINUX Redhat 4. Thanks (3 Replies)
Discussion started by: yoavbe
3 Replies

8. UNIX for Advanced & Expert Users

Jobs scheduled in crontabs are not running after a migration to oracle 9i

Good Morning. The problem started after tha migration to a newer version of oracle - migration to oracle 9i. Before the migration jobs that were scheduled in crontabs were running ok. but after the migration they are not running. I understand that is not easy to find out what the problem is.... (4 Replies)
Discussion started by: alexcol
4 Replies

9. Solaris

monitoring cpu

I have a windows snmp server running prtg. I have a solaris 8 v440R server and only see the network statistics for the server. When I try to add a new service to watch for the CPU, I am not offered the results. There is the default sun.mib installed. SNMPDX is not running but mibiisa is. I... (1 Reply)
Discussion started by: csross
1 Replies

10. AIX

Monitoring

Hi Can any one helpe me in obtaining commands , which enables me to know the following : Monitor Storage Monitor RAM & CPU Utilization Monitor Power Supply Best wishes (1 Reply)
Discussion started by: magasem
1 Replies

Featured Tech Videos