cron script to rotate log files | Unix Linux Forums | Shell Programming and Scripting

  Go Back    


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

cron script to rotate log files

Shell Programming and Scripting


Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 04-06-2009
timgolding timgolding is offline
Registered User
 
Join Date: Dec 2008
Last Activity: 4 November 2013, 5:24 AM EST
Posts: 83
Thanks: 10
Thanked 2 Times in 2 Posts
cron script to rotate log files

I have a mac server. I have been having problems with my logs. My hard disk became full, when i researched into why it was full it was due to massive log files. There was barley any log rotation policies in place on the server. I tired to use logrotate. This doesn't work on my server. It is a MAC server and doesn't seems to have logrotate. However some of the log files on the server had been being rotated. I looked into why. I found a file that by default is run at 4:30 every Saturday. it is /etc/weekly. I opened the file and found some script that was doing the logrotate on the files that had been being rotated. The script looked like this.


Code:
echo ""
printf %s "Rotating log files:"
cd /var/log
for i in ftp.log lookupd.log lpr.log mail.log netinfo.log hwmond.log ipfw.log p$
    if [ -f "${i}" ]; then
        printf %s " $i"
        if [ -x /usr/bin/gzip ]; then gzext=".gz"; else gzext=""; fi
        if [ -f "${i}.3${gzext}" ]; then mv -f "${i}.3${gzext}" "${i}.4${gzext}$
        if [ -f "${i}.2${gzext}" ]; then mv -f "${i}.2${gzext}" "${i}.3${gzext}$
        if [ -f "${i}.1${gzext}" ]; then mv -f "${i}.1${gzext}" "${i}.2${gzext}$
        if [ -f "${i}.0${gzext}" ]; then mv -f "${i}.0${gzext}" "${i}.1${gzext}$
        if [ -f "${i}" ]; then mv -f "${i}" "${i}.0" && if [ -x /usr/bin/gzip ]$
        touch "${i}" && chmod 640 "${i}" && chown root:admin "${i}"
    fi
done
if [ -f /var/run/syslog.pid ]; then kill -HUP $(cat /var/run/syslog.pid | head $
echo ""

Cool I had found the script that did the log rotations. Bravely i decided to edit that script. I edited it. I ensured the following.
1. I could add my own log files to the for loop
2. Since the log files i was adding would have different permissions i would need to return octal file permissions, the user group and user of that log file. so i could use it in the touch, chmod and chown command
3. I wanted it to only rotate logs if the size of the log file was over 2MB. Otherwise not to bother.

So i designed my script to abide by those rules.
Here is my script


Code:
echo ""
printf %s "Rotating log files:"
cd /var/log
# Add new logs to this list
for i in daily_out.log asl.log monthly.out weekly.out lastlog windowserver.log SerialNumberSupport.log servermgrd.log rsync_backup.log crashreporter.log ftp.log lookupd.log lpr.log mail.log netinfo.log hwmond.log ipfw.log ppp.log secure.log; do
    if [ -f "${i}" ]; then
        #Mods by Tim Golding - shell noob. this mod makes sure logrotations only occure if file over 2M. Should allow not system logs to be added to the for loop.
        file_size=`ls -l "${i}"| awk '{printf "%s",$5}'`
        max=2097152
        if [ $file_size -gt $max ]; then
        #end of mods                                                                           
                ch_mod=`ls -l "${i}"| awk '{k=0;for(i=0;i<=8;i++)k+=((substr($1,i+2,1)~/[rwx]/)*2^(8-i));if(k)printf("%0o ",k);print}'| awk '{printf "%s",$1}'`
                this_group=`ls -l "${i}"| awk '{printf "%s",$3}'`
                this_user=`ls -l "${i}"| awk '{printf "%s",$4}'`
                printf %s " $i"
                if [ -x /usr/bin/gzip ]; then gzext=".gz"; else gzext=""; fi
                if [ -f "${i}.3${gzext}" ]; then mv -f "${i}.3${gzext}" "${i}.4${gzext}"; fi
                if [ -f "${i}.2${gzext}" ]; then mv -f "${i}.2${gzext}" "${i}.3${gzext}"; fi
                if [ -f "${i}.1${gzext}" ]; then mv -f "${i}.1${gzext}" "${i}.2${gzext}"; fi
                if [ -f "${i}.0${gzext}" ]; then mv -f "${i}.0${gzext}" "${i}.1${gzext}"; fi
                if [ -f "${i}" ]; then mv -f "${i}" "${i}.0" && if [ -x /usr/bin/gzip ]; then gzip -9 "${i}.0"; fi; fi
                touch "${i}" && chmod "${ch_mod}" "${i}" && chown "${this_group}":"${this_user}" "${i}"
        fi
    fi
done
if [ -f /var/run/syslog.pid ]; then kill -HUP $(cat /var/run/syslog.pid | head -1); fi
echo ""

So i wrote this script telling myself how clever i was. Until Saturday when the script run. There were only two log files over 2M. asl.log and weekly_out.log. They were both huges. the script had backed up the old logs gzipped the backups and created new logs with the correct permissions. But the new logs are 0 bytes and are not being written to. What have a done wrong. Is it something to do with not killing the process as is done in the bottom line of the script with syslog.pid. The bottom line is tired to improve the logs policies but have made the situation alot worse. Can anyone offer any advice?

Last edited by timgolding; 04-07-2009 at 06:11 AM..
Sponsored Links
    #2  
Old 04-07-2009
timgolding timgolding is offline
Registered User
 
Join Date: Dec 2008
Last Activity: 4 November 2013, 5:24 AM EST
Posts: 83
Thanks: 10
Thanked 2 Times in 2 Posts
Anyone?
Sponsored Links
    #3  
Old 04-14-2009
timgolding timgolding is offline
Registered User
 
Join Date: Dec 2008
Last Activity: 4 November 2013, 5:24 AM EST
Posts: 83
Thanks: 10
Thanked 2 Times in 2 Posts
Please someone help me
    #4  
Old 04-14-2009
pludi's Avatar
pludi pludi is offline Forum Advisor  
Cat herder
 
Join Date: Dec 2008
Last Activity: 28 March 2014, 8:35 AM EDT
Location: Vienna, Austria, Earth
Posts: 5,522
Thanks: 38
Thanked 335 Times in 308 Posts
Without any more information on the services that write to those logs I'll take a wild guess and say that these still have an open handle on the files you're rotating. Usually there's a mechanism like in the second-to-last line, sending a signal (usually SIGHUP or SIGUSR1) to the process, telling it to re-read the configuration and/or recreating the log files.

If there's no such thing, you'll probably have to restart the daemon.
Sponsored Links
    #5  
Old 04-14-2009
timgolding timgolding is offline
Registered User
 
Join Date: Dec 2008
Last Activity: 4 November 2013, 5:24 AM EST
Posts: 83
Thanks: 10
Thanked 2 Times in 2 Posts
Thankyou for that information. Here is a list of log files that i added

Quote:
daily_out.log
asl.log
monthly.out
weekly.out
lastlog
windowserver.log
SerialNumberSupport.log
servermgrd.log
rsync_backup.log
crashreporter.log
ppp.log
secure.log
How can i find out which process has a handle open to that log file? If i can find out which deamon uses which log I can at least research into what the mechanism might be for each one?
Sponsored Links
    #6  
Old 04-14-2009
pludi's Avatar
pludi pludi is offline Forum Advisor  
Cat herder
 
Join Date: Dec 2008
Last Activity: 28 March 2014, 8:35 AM EDT
Location: Vienna, Austria, Earth
Posts: 5,522
Thanks: 38
Thanked 335 Times in 308 Posts
These look like something generated by cron or similar:
  • daily_out.log
  • monthly.out
  • weekly.out
This is used by the login management system and should never be rotated or otherwise modified manually:
  • lastlog
Probably used by the UI and/or for support issues:
  • windowserver.log
  • SerialNumberSupport.log
  • crashreporter.log
Probably generated by a script:
  • rsync_backup.log
No clue:
  • asl.log
  • servermgrd.log
  • ppp.log
  • secure.log

Quote:
Originally Posted by timgolding View Post
[...]
How can i find out which process has a handle open to that log file? If i can find out which deamon uses which log I can at least research into what the mechanism might be for each one?
Just look through the config files for the daemons you're using to check what files they're writing to. And the mechanism needed is usually noted in the man page (or other shipped documentation). If there's no explicit information about signal involved, better play it save and just restart the daemon.

Last edited by pludi; 04-14-2009 at 10:02 AM.. Reason: Better readability
Sponsored Links
    #7  
Old 04-14-2009
vgersh99's Avatar
vgersh99 vgersh99 is online now Forum Advisor  
Forum Advisor
 
Join Date: Feb 2005
Last Activity: 18 December 2014, 10:09 PM EST
Location: Foxborough, MA
Posts: 7,679
Thanks: 156
Thanked 591 Times in 555 Posts
look into 'man fuser' - it will give you the PIDs of the processes using the files.
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Script for Log Rotate El Rengo Shell Programming and Scripting 1 12-24-2008 06:43 AM
Help with script, trying to get tcpdump and rotate the file every 300 seconds livewire Shell Programming and Scripting 3 09-10-2008 11:45 AM
Perl script to rotate logs theninja Shell Programming and Scripting 18 03-21-2008 04:06 PM
how to rotate log files satya_skm Shell Programming and Scripting 0 03-08-2008 09:51 PM
Check size and rotate log script. ashishT HP-UX 3 12-16-2005 06:43 PM



All times are GMT -4. The time now is 11:11 PM.