Login or Register to Ask a Question and Join Our Community


Red Hat

Need Script to ZIP/SAVE & then DELETE Log file & DELETE ZIPS older than 12 months

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems Linux Red Hat Need Script to ZIP/SAVE & then DELETE Log file & DELETE ZIPS older than 12 months
# 1  
Old 12-07-2012
Code Need Script to ZIP/SAVE & then DELETE Log file & DELETE ZIPS older than 12 months
ENVIROMENT
Linux: Fedora Core release 1 (Yarrow)
iPlanet: iPlanet-WebServer-Enterprise/6.0SP1
Log Path: /usr/iplanet/servers/https-company/logs

I have iPlanet log rotation enabled rotating files on a daily basis.

The rotated logs are NOT compressed & are taking up too much space.

I need a script that will run daily that will do 3 things:

1. Check the folder for uncompressed log files, compress them into a ZIP file
2. Delete the uncompressed log files after they are compressed
3. Check the folder & delete the ZIP log files that are OLDER than 12 months.
# 2  
Old 12-07-2012
Assumes zip is on your machine as /usr/bin/zip and you use sendmail (mailx is a front end for sendmail).

The while read loops are there to allow trapping errors.

Code:
#!/bin/bash
cd /usr/iplanet/servers/https-company/logs
# keep today's log files as text
find . -name '*.log' -mtime +1 -type f |
while read fname
do
   /usr/bin/zip ${fname}.zip $fname
   if [ ? -ne o ] ; then
      echo "fatal error compressing log $fname" | mailx -s 'Log Error' zachs@my.com
      exit 1
   fi
   rm $fname || echo "fatal error $fname" | mailx -s 'Log Error' zachs@my.com && exit 1
done
# delete old zip files after one year
find . -type f -name '*.zip' -mtime +365 | 
while read fname 
do
  rm $fname || echo "fatal error $fname" | mailx -s 'Log Error' zachs@my.com && exit 1
done
# good exit
echo "log maintenance completed" | mailx -s 'Logs ok' zachs@my.com
exit 0

# 3  
Old 12-10-2012
Thanks! & a quick Question
this post was removed because I "fixed" the issues
Last edited by zachs; 12-11-2012 at 11:26 AM..
# 4  
Old 12-11-2012
Followup on the *.log file name
The script was able to zip a file but i think it needs some help in finding the "log" file...?

because the access & error logs don't have the file extension noted in the script '*.log'

When I do a ls -l in that folder there are no file extensions on the log files.... I need to fix the script to find these files to compress & delete

Let me know what you think? Thanks!

PS - I got the email to send so we're almost there! Smilie

Code:
-rw-r--r-- 1 deals deals 1046826 Dec 11 08:59 access
-rw-r--r-- 1 deals deals 92941280 Dec 7 00:00 access.201212070000
-rw-r--r-- 1 deals deals 3044935 Dec 8 00:00 access.201212080000
-rw-r--r-- 1 deals deals 2711784 Dec 8 23:59 access.201212090000
-rw-r--r-- 1 deals deals 2653255 Dec 9 23:59 access.201212100000
-rw-r--r-- 1 deals deals 2688456 Dec 11 00:00 access.201212110000
drwxr-xr-x 2 root root 4096 Jul 24 2007 archive
-rw-r--r-- 1 deals deals 24136805 Dec 11 08:59 errors
-rw-r--r-- 1 deals deals 147018158 Dec 7 00:00 errors.201212070000
-rw-r--r-- 1 deals deals 88812144 Dec 7 23:59 errors.201212080000
-rw-r--r-- 1 deals deals 50790062 Dec 8 23:58 errors.201212090000
-rw-r--r-- 1 deals deals 47430403 Dec 9 23:59 errors.201212100000
-rw-r--r-- 1 deals deals 47537385 Dec 11 00:00 errors.201212110000

---------- Post updated at 10:28 AM ---------- Previous update was at 10:25 AM ----------

Code:
#!/bin/bash
cd /usr/iplanet/servers/https-company/logs
# keep today's log files as text
find . -name '*.log' -mtime +1 -type f |
while read fname
do
   /usr/bin/zip ${fname}.zip $fname
   if [ ? -ne o ] ; then
      echo "fatal error compressing log $fname" | /bin/mail -s 'Log Error' zachs@domain.com
      exit 1
   fi
   rm $fname || echo "fatal error $fname" | /bin/mail -s 'Log Error' zachs@domain.com && exit 1
done
# delete old zip files after one year
find . -type f -name '*.zip' -mtime +365 | 
while read fname 
do
  rm $fname || echo "fatal error $fname" | /bin/mail -s 'Log Error' zachs@domain.com && exit 1
done
# good exit
echo "log maintenance completed" | /bin/mail -s 'Logs ok' zachs@domain.com
exit 0

Last edited by Corona688; 12-11-2012 at 11:50 AM..
# 5  
Old 12-11-2012
If you're only putting one file in each zip, there's no point using old-fashioned pkware pkzip, you might as well use a stream compressor like gzip. gzip will also delete the file by itself if it succeeds (and not if it fails).

You don't need to use $?, you can put the command directly into the if-statement.

I'm also not convinced your || && chain does precisely what you want, I'd make it an if.

This should match any log files ending in a dot then a few numbers, and avoid already-compressed logs.

Code:
#!/bin/bash
cd /usr/iplanet/servers/https-company/logs
# keep today's log files as text
find . -name '*.[0-9]*' '!' -name '*.gz' -mtime +1 -type f |
while read fname
do
    if ! gzip "$fname"
    then
        echo "fatal error compressing log $fname" | /bin/mail -s 'Log Error' zachs@domain.com
        exit 1
   fi
done

# delete old zip files after one year
find . -type f -name '*.gz' -mtime +365 | 
while read fname 
do
        if ! rm $fname
        then
                echo "fatal error $fname" | /bin/mail -s 'Log Error' zachs@domain.com
                exit 1
        fi
done
# good exit
echo "log maintenance completed" | /bin/mail -s 'Logs ok' zachs@domain.com
exit 0

# 6  
Old 12-12-2012
You just want this in a zip, right. Nothing fancy?
# 7  
Old 12-12-2012
correct in a zip nothing fancy at all.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SFTP Shell Script Get & Delete && Upload & Delete

Hi All, Do you have any sample script, - auto get file from SFTP remote server and delete file in remove server after downloaded. - only download specify filename - auto upload file from local to SFTP remote server and delete local folder file after uploaded - only upload specify filename ... (3 Replies)
Discussion started by: weesiong
3 Replies

2. UNIX for Dummies Questions & Answers

Log file - Delete duplicate line & keep last date

Hello All ! I need your help on this case, I have a csv file with this: ITEM105;ARI FSR;2016-02-01 08:02;243 ITEM101;ARI FSR;2016-02-01 06:02;240 ITEM032;RNO TLE;2016-02-01 11:03;320 ITEM032;RNO TLE;2016-02-02 05:43;320 ITEM032;RNO TLE;2016-02-01 02:03;320 ITEM032;RNO... (2 Replies)
Discussion started by: vadim-bzh
2 Replies

3. Shell Programming and Scripting

Need Script to ZIP/SAVE & then DELETE Log file & send a mail conformation for any error

ENVIROMENT Linux: RHEL 6.4 Log Path: /usr/iplanet/servers/https-company/logs Log Format: user.log.03-15-2015 I have log4j log rotation enabled rotating files on a daily basis. The rotated logs are NOT compressed & are taking up too much space. I need a script that will run daily that... (1 Reply)
Discussion started by: admin_job_admin
1 Replies

4. Shell Programming and Scripting

Script needed to delete to the list of files in a directory based on last created & delete them

Hi My directory structure is as below. dir1, dir2, dir3 I have the list of files to be deleted in the below path as below. /staging/retain_for_2years/Cleanup/log $ ls -lrt total 0 drwxr-xr-x 2 nobody nobody 256 Mar 01 16:15 01-MAR-2015_SPDBS2 drwxr-xr-x 2 root ... (2 Replies)
Discussion started by: prasadn
2 Replies

5. Shell Programming and Scripting

[Solved] Get files & delete them by shell script

I want to use my script to get any file then delete it once it transfers to my side , I manage to create below script to generate "list" file which contains all file names in "10.10.1.1" then I made "a.out" file which contains the commands that I want to run it on "10.10.1.1" to get & delete the... (2 Replies)
Discussion started by: arm
2 Replies

6. Shell Programming and Scripting

help with tar & zip only last months(say,Sep) files

Need to 1. archive all the files in a directory from the previous month into a tar/gz file, ignoring all already archived 'tar.gz' files 2. Check created .tar.gz file isnt corrupted and has all the required files in it. and then remove the original files. I am using a function to get the... (1 Reply)
Discussion started by: Prev
1 Replies

7. Shell Programming and Scripting

Shell script delete log files from folder & subfolders on space usage

Hi, I am trying to write a shell script to delete logs generate by db when space in the folder reaches 70%. i am getting space values from db, find the files at OS and remove them by using a cron job runs every 5minutes. I have to keep the latest 5 files at any time, my problem is that log files... (3 Replies)
Discussion started by: saha
3 Replies

8. Shell Programming and Scripting

Delete files older than 3 months.(read date from the name of the file)

Guys, My log files stored in the date format format below(log_20080714072942): TIMESTAMP=`date +%Y%m%d%H%M%S` LOG=/log/log_${TIMESTAMP}.log I'm looking for a shell script which deletes all files which is older than 3 months from today. Regards, Bhagat (3 Replies)
Discussion started by: bhagat.singh-j
3 Replies

9. UNIX for Advanced & Expert Users

how to archive logs older than 5 days & then delete them?

My code is tar -cvf logs.tar `find /usr/openv/logs/512*.log -mtime +2` && find *.log* -mtime +2 -exec rm {} \; this gives me output as: tar: Missing filenames:confused: (1 Reply)
Discussion started by: timus1980
1 Replies

10. Shell Programming and Scripting

Overwrite & Delete in Text File

Dear All, I have text file like this: Header Record 1 Record 2 ....... Record n Tail This line of code : awk '{ if ( NR == 1 ) { head=substr($0,1,300);} else { last = substr($0,1,300);}END{printf "Header is : %-300s Trailer is : %-300s\n", head, last}' filename converted Header... (11 Replies)
Discussion started by: 33junaid
11 Replies
Login or Register to Ask a Question