cron/logrotate chicken and egg problem Post: 302235278

Sponsored Content

Top Forums UNIX for Advanced & Expert Users cron/logrotate chicken and egg problem Post 302235278 by jjinno on Thursday 11th of September 2008 02:13:22 PM

09-11-2008

Registered User

cron/logrotate chicken and egg problem

I have run into a problem where about a dozen machines, all the same x86_64 2.6.12 GNU/Linux. For some reason these machines will fill up their /var partition (10G), because their logs never get rotated... Unfortunately, there is no error message from logrotate (would be in /var/log/messages) and the last time logrotate ran (according to /var/log/logrotate.log) was August 11, 2008.

This unfortunately is somewhat of a puzzling problem, making it feel (however unlikely) that this is actually a problem with cron, and not log-rotate. IE. cron failed first, which made logrotate never happen. I had originally thought that log-rotate just couldn't do its job because there was no space available for the rotation to occur. But even when I gave it enough space, cron just never ran the logrotate script. On top of that, I even added my own "append timestamp to file" script to the cron.hourly, and it never got run either. As a test, on one of the servers, I restarted cron... BINGO, it fixed the whole thing.

I could care less about the fact that I can fix it. I want to prevent this from ever happening in the field. In my lab is one thing, but in the field, there goes all my logs for debugging.

Here is a sanity check I ran (just proving that cron was functioning and stopped functioning - all within the same up-time):

Code:

[root@babylon5 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2             9.5G  3.7G  5.3G  42% /
/dev/sda1             190M   18M  163M  10% /boot
/dev/shm              2.0G  4.0K  2.0G   1% /dev/shm
/dev/sda6              44G  8.4G   33G  21% /stuff
/dev/sda5             9.5G  9.5G     0 100% /var

[root@babylon5 ~]# ls -lut /etc/init.d/crond
-rwxr-xr-x  1 root root 1904 Aug 11 15:14 /etc/init.d/crond

[root@babylon5 ~]# ls -lut /etc/crontab
-rw-r--r--  1 root root 255 Jul 15 10:19 /etc/crontab

[root@babylon5 ~]# grep "Aug 11 15" /var/log/cron
Aug 11 15:01:02 babylon5 crond[9429]: (root) CMD (run-parts /etc/cron.hourly)

[root@babylon5 ~]# ps auxwww|grep cr[o]n
root      3014  0.0  0.0  60404  1296 ?        Ss   Jul15   0:00 crond

[root@babylon5 ~]# uptime
 13:33:34 up 57 days,  3:15,  1 user,  load average: 15.63, 15.37, 15.29

[root@babylon5 ~]# grep "STARTUP" /var/log/cron*
/var/log/cron.4:Jul 14 14:16:44 babylon5 crond[3156]: (CRON) STARTUP (V5.0)
/var/log/cron.4:Jul 15 08:53:45 babylon5 crond[3008]: (CRON) STARTUP (V5.0)
/var/log/cron.4:Jul 15 10:19:35 babylon5 crond[3014]: (CRON) STARTUP (V5.0)

[root@babylon5 ~]# tail -n 1 /var/log/cron.4
Jul 20 00:01:01 babylon5 crond[15555]: (root) CMD (run-parts /etc/cron.hourly)

[root@babylon5 ~]# tail -n 1 /var/log/cron.3
Jul 27 01:01:02 babylon5 crond[27941]: (root) CMD (run-parts /etc/cron.hourly)

[root@babylon5 ~]# tail -n 1 /var/log/cron.2
Aug  3 01:01:01 babylon5 crond[17755]: (root) CMD (run-parts /etc/cron.hourly)

[root@babylon5 ~]# tail -n 1 /var/log/cron.1
Aug 10 01:01:02 babylon5 crond[29697]: (root) CMD (run-parts /etc/cron.hourly)

[root@babylon5 ~]# tail -n 1 /var/log/cron
Sep 10 13:01:01 babylon5 crond[22186]: (root) MAIL (mailed 146 bytes of output but got status 0x007f )

[root@babylon5 ~]# ll /var/log/cron*
-rw-r--r--  1 root root 138502 Sep 10 13:01 /var/log/cron
-rw-------  1 root root  14407 Aug 10 01:01 /var/log/cron.1
-rw-------  1 root root  14581 Aug  3 01:01 /var/log/cron.2
-rw-------  1 root root  14483 Jul 27 01:01 /var/log/cron.3
-rw-------  1 root root  14513 Jul 20 00:01 /var/log/cron.4

Any help?

jjinno

View Public Profile for jjinno

Find all posts by jjinno

8 More Discussions You Might Find Interesting

1. Red Hat

Problem seen with logrotate

Hi all, I have configured logrotate to logorotate every 12 hour. The configurations are as follows. /etc/cron.d/config ------------------------- SHELL=/bin/bash PATH=/sbin:/bin:/usr/sbin:/usr/bin MAILTO="" HOME=/root 0 */12 * * * root logrotate /etc/logrotate.d/test ...

2. UNIX for Advanced & Expert Users

logrotate with /etc/logrotate.conf file

Hi there, I want to rotate the logfiles which are located in /var/log/jboss/tomcat* so I have created a file named as 'tomat' in /etc/logrotate.d/tomcat with the following content. # cat /etc/logrotate.d/tomcat /var/log/jboss/tomcat_access_log*.log { daily nocreate ...

3. UNIX for Advanced & Expert Users

Logrotate configuration problem

Hi, I have the following configuration file: /logs/system/mindundi/* { rotate 0 daily missingok sharedscripts postrotate find /logs/system/mindundi/ -name "*log" -mtime +15 -exec /bin/rm -f {} \; endscript } I want to save only...

4. UNIX for Dummies Questions & Answers

logrotate and cron.daily/weekly

Hi guys, I've got two separate logrotates I'd like to run, one for Tomcat and one for Apache, but I'd like to run the Tomcat one daily and the Apache one weekly. Now, the logrotate itself is working fine, but although I have 'daily' in Tomcat, and 'weekly' in the Apache one, the latter is...

5. Shell Programming and Scripting

Logrotate - I am not able to rotate files using logrotate

I have written script which is working in Home directory perfectly and also compressing log files and rotating correctly. But, when i try to run script for /var/log/ i am able to get compressed log files but not able to get rotation of compressed log files. Please suggest. I am using below command...

6. AIX

Logrotate - /etc/logrotate.conf does't exist

Hi Admins. I have installed logrotate rpm on Aix 6.1. After the installation of rpm, I don't find /etc/logrotate.conf file and /etc/logrotate.d dir . The config file is located in /opt/freeware/etc/logrotate.conf. When I ran logrotate -v /opt/freeware/etc/logrotate.conf I get below...

7. Shell Programming and Scripting

Problem with logrotate

Hi, I have a problem with logrotate at Centos 7. My logrotate is configured with "rotate 0" to Apache logs, so it should never keep logs when rotating, just removing them and replacing by new empty ones at every rotation. But for some reason, once in a while, I see that logrotate is creating...

8. Shell Programming and Scripting

How to disable cron emails, but only for logrotate only not for other logs?

Guys, is there a script or command? how to disable cron emails, but only for logrotate only not for other logs

LEARN ABOUT DEBIAN

apticron

apticron(1)						      General Commands Manual						       apticron(1)

NAME

       apticron - Generate a mail listing packages which are pending an upgrade

SYNOPSIS

       apticron [--cron]

DESCRIPTION

       apticron is a shell script which generates a mail with a list of all packages currently pending an upgrade, as well as summary of changes -
       using apt-listchanges - to a configurable Email address.

       apticron is mainly intended for automatic notification of pending security updates but can also be used	in  many  other  situations  where
       timely updates are necessary.

       The --cron option is only meant to be used when invoked by cron in order to run no more than once a day.

FILES

       /etc/apticron/apticron.conf
	      Configuration file, Email address to mail and profile for use with apt-listchanges are configurable.

       /etc/cron.d/apticron
	      Cron job for executing apticron daily.

       /etc/cron.daily/apticron
	      Fallback cron script for executing apticron (if no crond running or system down at designated time).

       /var/lib/misc/apticron.cron
	      Time stamp file used by apticron when invoked with --cron option.

SEE ALSO

       apt-get(8), apt-listchanges(1), cron(8)

AUTHOR

       Apticron  was initially developed by Colm MacCarthaigh <colm.maccarthaigh@heanet.ie> with contributions from Marc Sherman <msherman@projec-
       tile.ca>. Since 2006 it is maintained and improved by Tiago Bortoletto Vaz <tiago@debian.org>.

																       apticron(1)