Sponsored Content
Full Discussion: Perl script to rotate logs
Top Forums Shell Programming and Scripting Perl script to rotate logs Post 302174892 by theninja on Wednesday 12th of March 2008 12:17:02 PM
Old 03-12-2008
Perl script to rotate logs

I have a shell script that will gzip/tar/archive application logs that are over 20 days old which works just fine, but I would like to convert to a Perl script. Problem is, I'm a beginner with Perl and all attempts so far have failed.
Basicaly I have a log dir /app/logs that contains several different logs files that are created daily. One log for each day.
this.log.2008.02.01
this.log.2008.02.02
this.log.2008.02.03
that.log.2008.02.01
that.log.2008.02.02
that.log.2008.02.03
and so on......

the script finds any log over 20 days old, gzips it, the moves it to /app/logs/archive, after 60 days these files are tar'd up and moved to /log_archive/app_logs

Any help would be greatly appreciated.

here is the shell script.....
Code:
#!/usr/bin/ksh

# Gzip is located in /usr/contrib/bin
export PATH=$PATH:/usr/contrib/bin

# Last 2 digits of the current year
curr_year=`date +%o`

# Directorys
LOG_DIR=/app/logs
AUDIT_DIR=/app/logs/audit
ARCHIVE_DIR=/app/logs/archive
LOG_ARCHIVE_DIR=/log_archive/app_logs

# Number of days before things are gzip'd.
COMPRESS_DAYS=7
# Number of days till gzips are added to a tar file.
MV_AR_ARCHIVE_DAYS=7
# Number of days until the tar files are moved to log partition.
MV_MOAR_ARCHIVE_DAYS=60
# Number of days till tars are removed from log partition.
RM_MOAR_ARCHIVE_DAYS=180

#############

MOAR_HOST_NAME=$(hostname|tr [:lower:] [:upper:])

COMPAREYEAR="`date '+%y'`"
DATEYEAR="`date '+%Y'`"
DATEMONTH="`date '+%m'`"
DATEMONTH=$(( $DATEMONTH - 1 ))

if [ $DATEMONTH -eq 0 ]
then
        DATEMONTH=12
        DATEYEAR=$(( $DATEYEAR - 1 ))
fi
if [ $DATEMONTH -lt 10 ]
then
        DATEMONTH=0$DATEMONTH
fi

# DATESTAMP is the last month
DATESTAMP=$DATEYEAR.$DATEMONTH
# echo $DATESTAMP

#change to the log directory
cd ${LOG_DIR}
echo "Compressing Logs files"
find *fin*log* -prune -mtime +${COMPRESS_DAYS} -exec gzip {} \;
find this* -prune -mtime +${COMPRESS_DAYS} -exec gzip {} \;
find that* -prune -mtime +${COMPRESS_DAYS} -exec gzip {} \;
#move GZIP files to archive directory
find *.gz -mtime +${MV_AR_ARCHIVE_DAYS} -exec mv {} ${LOG_ARCHIVE_DIR}/ \;

#move GZIP files to archive directory
find *.gz -mtime +${MV_AR_ARCHIVE_DAYS} -exec mv {} ${ARCHIVE_DIR}/  \;

#change to the Archive Directory
cd ${ARCHIVE_DIR}
echo "Adding GZIP files to tar archive"
if [ ! -f "arlogs_${DATESTAMP}.tar" ]
then
        touch place.txt
        tar -cf arlogs_${DATESTAMP}.tar place.txt
fi

#loop for all the files in the directory.
ls -l *gz |\
while read file; do
    # Get month
    mo=`echo $file |awk '{print $6}'`
    case $mo in
        Jan) month=01;;
        Feb) month=02;;
        Mar) month=03;;
        Apr) month=04;;
        May) month=05;;
        Jun) month=06;;
        Jul) month=07;;
        Aug) month=08;;
        Sep) month=09;;
        Oct) month=10;;
        Nov) month=11;;
        Dec) month=12;;
          *) echo "Error:$prog:cannot determine month"; exit 1;;
    esac

    # Get day
    day=`echo $file |awk '{print $7}'`
    if [ -z "$day" ]; then
        echo "Error:$prog:cannot determine day"
        exit 1
    fi
    if [ $day -lt 10 ]; then
        day="0$day"
    fi

    # Get year
    file_year=`echo $file |awk '{print $8}'`
    if [ `echo $file_year |grep -c ":"` -gt 0 ]; then
        year=$curr_year
    else
        year=`echo $file_year |sed 's/^..//'`
    fi
    if [ -z "$year" ]; then
        echo "Error:$prog:cannot determine year"
        exit 1
    fi

    # File modification time (timestamp) in xx/xx/xx format
    timestamp="$month/$day/$year"

    # Get filename from long listing
    file=`echo $file |awk '{print $9}'`

    # Add to tar file and remove gzip file
    case $timestamp in
        *$DATESTAMP*)
            tar -uf arlogs_${DATESTAMP}.tar ${file}
            if [ $? -eq 0 ]; then
                rm ${file}
            fi ;;
        $DATEMONTH/*/$COMPAREYEAR)
            tar -uf arlogs_${DATESTAMP}.tar ${file}
            if [ $? -eq 0 ]; then
                rm ${file}
            fi ;;
    esac
done

#check to see if directory on archive partition is there
if [ ! -d "${LOG_ARCHIVE_DIR}/${MOAR_HOST_NAME}_logs_${DATEYEAR}" ]
then
  mkdir ${LOG_ARCHIVE_DIR}/${MOAR_HOST_NAME}_logs_${DATEYEAR}
fi

# Move from the archive directory to the archive partition
find *.tar -mtime +${MV_MOAR_ARCHIVE_DAYS} -exec mv {} ${ARCHIVE_DIR}/${MOAR_HOST_NAME}_logs_${DATEYEAR} \;

#change to the archive partition directory
cd ${LOG_ARCHIVE_DIR}/${MOAR_HOST_NAME}_logs_${DATEYEAR}

# remove any old tars
find *.tar -mtime +${RM_MOAR_ARCHIVE_DAYS} -exec rm {} \;


Thanks
Theninja
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script for Log Rotate

Hello, I only know the basic for shell programing. I need help for this, I thinks this is a basic for anyone who know a litle of shell scripting. I need creat a script for a rotatate logs, when a filesystem is full. I have a filesystem. The rotate consist in zip the current log (copy) and... (1 Reply)
Discussion started by: El Rengo
1 Replies

2. UNIX for Dummies Questions & Answers

Rotate logs every 1 hour

Hello All, I am learning unix and basically I want to rotate one of my application logs every 1 hour. I need to rotate that file every one hour. I looked in the forums and googled.. but couldn;t get proper information. Requesting you all to kindly guide me. Our application is running on... (4 Replies)
Discussion started by: arunpvp
4 Replies

3. Shell Programming and Scripting

Help with a rotate log script

Hi all, Am trying to write my own log rotate script. Curremtly, what I have is as below: #!/bin/ksh file_to_rotate=${1} x=${2} while ] do let curr=${x} let prev=${x}-1 if ] ; then #echo "cp -p ${file_to_rotate} ${file_to_rotate}.${curr}" cp -p... (7 Replies)
Discussion started by: newbie_01
7 Replies

4. Shell Programming and Scripting

Script to rotate file log

Hi Experts, I have script on crontab and give output quite large. I would like to know how to create rotate log when the size of log maximum 50MB if the test.log is 50MB then create test.0 Thanks Edy (2 Replies)
Discussion started by: edydsuranta
2 Replies

5. Shell Programming and Scripting

Perl script to parse multiple windows event logs.

Hi all, I am developing a log parsing agent in perl to send windows Event logs to Zenoss Monitoring tool. Using Win32::EventLog i can able to get the Event messages but only one Eventype eg Application or System could able to parse at a time. Can you please help to how to open mutiple eventlogs... (3 Replies)
Discussion started by: kar_333
3 Replies

6. Shell Programming and Scripting

Perl script to extract last 24 hrs logs from cronlog

Hi Friends, Can anybody help me to create a perl script to generate log file for last 24 hrs from cron log file ?? Thank You (2 Replies)
Discussion started by: syamji.vm
2 Replies

7. UNIX for Dummies Questions & Answers

Need script to rotate logs

I have few solaris-10 non global zones, where one application is writing some logs to /var/ovd/ConfigLogs. It keeps increasing all the time, as it is needed by application team as of now. I want a small script, which I can configure in cronjob, which should : - Run every Saturday 10 PM - Copy... (5 Replies)
Discussion started by: solaris_1977
5 Replies

8. UNIX for Dummies Questions & Answers

Logs do not rotate

My problem: Both access and error logs do not rotate any more and get really large. They are located here: /srv/www/+vHost name here+/logs/ Configuration seems to be here: /etc/logrotate.conf => looks OK, including "size 10M" to avoid large files (/etc/logrotate.d => is empty) manually... (4 Replies)
Discussion started by: floko
4 Replies

9. Shell Programming and Scripting

If I ran perl script again,old logs should move with today date and new logs should generate.

Appreciate help for the below issue. Im using below code.....I dont want to attach the logs when I ran the perl twice...I just want to take backup with today date and generate new logs...What I need to do for the below scirpt.............. 1)if logs exist it should move the logs with extention... (1 Reply)
Discussion started by: Sanjeev G
1 Replies
All times are GMT -4. The time now is 07:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy