Visit Our UNIX and Linux User Community


cron script to rotate log files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting cron script to rotate log files
# 1  
Old 04-06-2009
cron script to rotate log files

I have a mac server. I have been having problems with my logs. My hard disk became full, when i researched into why it was full it was due to massive log files. There was barley any log rotation policies in place on the server. I tired to use logrotate. This doesn't work on my server. It is a MAC server and doesn't seems to have logrotate. However some of the log files on the server had been being rotated. I looked into why. I found a file that by default is run at 4:30 every Saturday. it is /etc/weekly. I opened the file and found some script that was doing the logrotate on the files that had been being rotated. The script looked like this.

Code:
echo ""
printf %s "Rotating log files:"
cd /var/log
for i in ftp.log lookupd.log lpr.log mail.log netinfo.log hwmond.log ipfw.log p$
    if [ -f "${i}" ]; then
        printf %s " $i"
        if [ -x /usr/bin/gzip ]; then gzext=".gz"; else gzext=""; fi
        if [ -f "${i}.3${gzext}" ]; then mv -f "${i}.3${gzext}" "${i}.4${gzext}$
        if [ -f "${i}.2${gzext}" ]; then mv -f "${i}.2${gzext}" "${i}.3${gzext}$
        if [ -f "${i}.1${gzext}" ]; then mv -f "${i}.1${gzext}" "${i}.2${gzext}$
        if [ -f "${i}.0${gzext}" ]; then mv -f "${i}.0${gzext}" "${i}.1${gzext}$
        if [ -f "${i}" ]; then mv -f "${i}" "${i}.0" && if [ -x /usr/bin/gzip ]$
        touch "${i}" && chmod 640 "${i}" && chown root:admin "${i}"
    fi
done
if [ -f /var/run/syslog.pid ]; then kill -HUP $(cat /var/run/syslog.pid | head $
echo ""

Cool I had found the script that did the log rotations. Bravely i decided to edit that script. I edited it. I ensured the following.
1. I could add my own log files to the for loop
2. Since the log files i was adding would have different permissions i would need to return octal file permissions, the user group and user of that log file. so i could use it in the touch, chmod and chown command
3. I wanted it to only rotate logs if the size of the log file was over 2MB. Otherwise not to bother.

So i designed my script to abide by those rules.
Here is my script

Code:
echo ""
printf %s "Rotating log files:"
cd /var/log
# Add new logs to this list
for i in daily_out.log asl.log monthly.out weekly.out lastlog windowserver.log SerialNumberSupport.log servermgrd.log rsync_backup.log crashreporter.log ftp.log lookupd.log lpr.log mail.log netinfo.log hwmond.log ipfw.log ppp.log secure.log; do
    if [ -f "${i}" ]; then
        #Mods by Tim Golding - shell noob. this mod makes sure logrotations only occure if file over 2M. Should allow not system logs to be added to the for loop.
        file_size=`ls -l "${i}"| awk '{printf "%s",$5}'`
        max=2097152
        if [ $file_size -gt $max ]; then
        #end of mods                                                                           
                ch_mod=`ls -l "${i}"| awk '{k=0;for(i=0;i<=8;i++)k+=((substr($1,i+2,1)~/[rwx]/)*2^(8-i));if(k)printf("%0o ",k);print}'| awk '{printf "%s",$1}'`
                this_group=`ls -l "${i}"| awk '{printf "%s",$3}'`
                this_user=`ls -l "${i}"| awk '{printf "%s",$4}'`
                printf %s " $i"
                if [ -x /usr/bin/gzip ]; then gzext=".gz"; else gzext=""; fi
                if [ -f "${i}.3${gzext}" ]; then mv -f "${i}.3${gzext}" "${i}.4${gzext}"; fi
                if [ -f "${i}.2${gzext}" ]; then mv -f "${i}.2${gzext}" "${i}.3${gzext}"; fi
                if [ -f "${i}.1${gzext}" ]; then mv -f "${i}.1${gzext}" "${i}.2${gzext}"; fi
                if [ -f "${i}.0${gzext}" ]; then mv -f "${i}.0${gzext}" "${i}.1${gzext}"; fi
                if [ -f "${i}" ]; then mv -f "${i}" "${i}.0" && if [ -x /usr/bin/gzip ]; then gzip -9 "${i}.0"; fi; fi
                touch "${i}" && chmod "${ch_mod}" "${i}" && chown "${this_group}":"${this_user}" "${i}"
        fi
    fi
done
if [ -f /var/run/syslog.pid ]; then kill -HUP $(cat /var/run/syslog.pid | head -1); fi
echo ""

So i wrote this script telling myself how clever i was. Until Saturday when the script run. There were only two log files over 2M. asl.log and weekly_out.log. They were both huges. the script had backed up the old logs gzipped the backups and created new logs with the correct permissions. But the new logs are 0 bytes and are not being written to. What have a done wrong. Is it something to do with not killing the process as is done in the bottom line of the script with syslog.pid. The bottom line is tired to improve the logs policies but have made the situation alot worse. Can anyone offer any advice?

Last edited by timgolding; 04-07-2009 at 06:11 AM..
# 2  
Old 04-07-2009
Anyone?
# 3  
Old 04-14-2009
Please someone help me Smilie
# 4  
Old 04-14-2009
Without any more information on the services that write to those logs I'll take a wild guess and say that these still have an open handle on the files you're rotating. Usually there's a mechanism like in the second-to-last line, sending a signal (usually SIGHUP or SIGUSR1) to the process, telling it to re-read the configuration and/or recreating the log files.

If there's no such thing, you'll probably have to restart the daemon.
# 5  
Old 04-14-2009
Thankyou for that information. Here is a list of log files that i added

Quote:
daily_out.log
asl.log
monthly.out
weekly.out
lastlog
windowserver.log
SerialNumberSupport.log
servermgrd.log
rsync_backup.log
crashreporter.log
ppp.log
secure.log
How can i find out which process has a handle open to that log file? If i can find out which deamon uses which log I can at least research into what the mechanism might be for each one?
# 6  
Old 04-14-2009
These look like something generated by cron or similar:
  • daily_out.log
  • monthly.out
  • weekly.out
This is used by the login management system and should never be rotated or otherwise modified manually:
  • lastlog
Probably used by the UI and/or for support issues:
  • windowserver.log
  • SerialNumberSupport.log
  • crashreporter.log
Probably generated by a script:
  • rsync_backup.log
No clue:
  • asl.log
  • servermgrd.log
  • ppp.log
  • secure.log

Quote:
Originally Posted by timgolding
[...]
How can i find out which process has a handle open to that log file? If i can find out which deamon uses which log I can at least research into what the mechanism might be for each one?
Just look through the config files for the daemons you're using to check what files they're writing to. And the mechanism needed is usually noted in the man page (or other shipped documentation). If there's no explicit information about signal involved, better play it save and just restart the daemon.

Last edited by pludi; 04-14-2009 at 10:02 AM.. Reason: Better readability
# 7  
Old 04-14-2009
look into 'man fuser' - it will give you the PIDs of the processes using the files.

Previous Thread | Next Thread
Test Your Knowledge in Computers #158
Difficulty: Easy
The migration of the ARPANET to TCP/IP was officially completed on January 1, 1983, when the new protocols were permanently activated.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Log rotate

Hi, I have below script in logrotate.d to rotate logs. logs are not rotating after the file grow to 1k, do you have any idea? Is it because of it just only 1K? Please let me know if the below syntax is in correct. # more trotate /sourcepath/*/servers/*/logs/*log... (2 Replies)
Discussion started by: lpprasad321
2 Replies

2. Shell Programming and Scripting

Script to rotate file log

Hi Experts, I have script on crontab and give output quite large. I would like to know how to create rotate log when the size of log maximum 50MB if the test.log is 50MB then create test.0 Thanks Edy (2 Replies)
Discussion started by: edydsuranta
2 Replies

3. Shell Programming and Scripting

Help with a rotate log script

Hi all, Am trying to write my own log rotate script. Curremtly, what I have is as below: #!/bin/ksh file_to_rotate=${1} x=${2} while ] do let curr=${x} let prev=${x}-1 if ] ; then #echo "cp -p ${file_to_rotate} ${file_to_rotate}.${curr}" cp -p... (7 Replies)
Discussion started by: newbie_01
7 Replies

4. Shell Programming and Scripting

"Customized" log rotate script - some advice please

Hi, Am trying to write my own version of a log rotate scripts 'coz I don't have the logrotate for other flavors of *nix servers. Probably should try and download the source and re-compile but our SA don't want to do so. Anyway, just want to know if there is any way to improve on the code... (0 Replies)
Discussion started by: newbie_01
0 Replies

5. Shell Programming and Scripting

Rotate log everyday on existing script

quick question: I have a current script that will put the output on a log file. See snapshot of the code below. I wanted this to be rotated everyday based on date. So if anyone execute the script today there will be a filecreated such as sys.log.(datetoday), if tomorrow it would be... (1 Reply)
Discussion started by: lhareigh890
1 Replies

6. Shell Programming and Scripting

Rotate log files

I have a big log,separated by the character:, one of the fields is the date in the format "day / month / year" and I need to remove the lines prior to 30 days. Can help me? (7 Replies)
Discussion started by: msanbrug
7 Replies

7. Shell Programming and Scripting

Script for Log Rotate

Hello, I only know the basic for shell programing. I need help for this, I thinks this is a basic for anyone who know a litle of shell scripting. I need creat a script for a rotatate logs, when a filesystem is full. I have a filesystem. The rotate consist in zip the current log (copy) and... (1 Reply)
Discussion started by: El Rengo
1 Replies

8. UNIX for Advanced & Expert Users

log rotate

hi , what is the meaning of log rotate? how do i rotate /var/adm/wtmps log and gzip it? (6 Replies)
Discussion started by: cromohawk
6 Replies

9. Shell Programming and Scripting

how to rotate log files

hi friends i need a shell script to rotate the logs in a directory, dated n days back. can anybody of help. appreciate.. (0 Replies)
Discussion started by: satya_skm
0 Replies

10. HP-UX

Check size and rotate log script.

Hi Can you suggest some perl script. My OS is HP-UX 11.11 I want to it into a cron job. Every night it will backup the file with that day's date and open a dummy file. Thanks Ash (3 Replies)
Discussion started by: ashishT
3 Replies

Featured Tech Videos