zipping older files with cron jobs


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting zipping older files with cron jobs
# 1  
Old 09-07-2012
zipping older files with cron jobs

Hi All,

In my team we generate huge logs and many a times due to this our total system crumbles.

I want to write a script which compares the file modification time with the current time and then if the difference is more than 'n' days the file is zipped , if the difference is more than ' n' days the file is deleted.

I want to schedule this at 1 am everyday and an email to be send to a main group whether the job was a success or not.

Please help its urgent.

Please use simple Unix commands if possible in the script.


Thanks
# 2  
Old 09-08-2012
Code:
###Go to the folder which contains log files
cd /root/Desktop/Scripts/logs
ls -lt > /root/Desktop/Scripts/logdate.txt


###Filtering the files with date
cat /root/Desktop/Scripts/logdate.txt | cut -d" " -f8  | sed  "/^$/d" > /root/Desktop/Scripts/date.txt


###Filtering the files with filename it will be usefull in the end
cat /root/Desktop/Scripts/logdate.txt | cut -d" " -f10  | sed  "/^$/d" > /root/Desktop/Scripts/filename.txt


a=`cat /root/Desktop/Scripts/date.txt`
b=`date | cut -d" " -f4`
for i in $a
do
##Comparing todays date with the files date and if it is lesserthan equal to 3 do the condition
if [ $(echo $b-$i | bc) -le 3  ]
then
logfile=`cat /root/Desktop/Scripts/filename.txt | sed -n '1p'`
##file is last modified with in last 3 days then zip the file
zip -9 ../$logfile.zip /root/Desktop/Scripts/logs/$logfile 
sed -i '1d' /root/Desktop/Scripts/filename.txt
else
##if the file is modified before 4 days or more delete it
logfile=`cat ../filename.txt | sed -n '1p'`
rm -rf $logfile
sed -i '1d' /root/Desktop/Scripts/filename.txt
fi
done
echo "Log file script runs successfully" | mail -s "script runs successfully" xyz@gmail.com

Cronjob
Code:
0 1 * * * /bin/sh /root/Desktop/Scripts/script.sh


Moderator's Comments:
Mod Comment Please view this code tag video for how to use code tags when posting code and data.

Last edited by Franklin52; 09-09-2012 at 11:18 AM..
This User Gave Thanks to rajkumarin For This Post:
# 3  
Old 09-08-2012
You don't mention what system (machine, OS) you are working on, but the find command usually does a pretty good job in finding files newer or older than some date. Some implementations allow immediate deletetion of files, and some need a bit tricky approach to the date, but they will do, and they all allow to execute a command like gzip or so on the files found.

In crontab, you may need to adapt the PATH variable to your needs, but mailing to a group is quite easy (again: if your implementation allows):
man crontab:
Quote:
If MAILTO is defined (and non-empty), mail is sent to the user so named. MAILTO may also be used to direct mail to multiple recipients by separating recipient users with a comma.
# 4  
Old 09-12-2012
Thnx.

@RajKumarIn: cut -d" " -f8 from the output of ls command is not giving required result as the file size varies and due to this the delimiter doesnt remain a single blank space.

Is there any way to trip the output for extra blank spaces so that i have a single space as delimiter?

@RudiC:

I am using centOS

Linux version 2.6.18-308.4.1.el5 (mockbuild@builder10.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-52))
# 5  
Old 09-12-2012
On linux the following might work:
Code:
find .  \( -newermt "12-09-10" ! -newermt "12-09-12" -ls \) , \( ! -newermt "12-09-09" -print \)

the first branch will ls all files modified between Sep 10 and Sep 12, in which you can replace -ls by -exec gzip {} \;, and the second prints all older files; here you could put a -ls -delete to list and then delete the files.
In your crontab, immediatedly before the entry for your script, put the
Code:
MAILTO user1@dom1,user2@dom2,...

statement.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Solaris

Cron jobs and at jobs

There are two jobs in Solaris , Cron and at jobs.. I know how to disable or enable cron jobs. How can I enable at jobs and disable it. Kindly help. Rj (2 Replies)
Discussion started by: jegaraman
2 Replies

2. Shell Programming and Scripting

Help with cron jobs

Hi Frenz, How do we get a cron job running in background to foreground ? (3 Replies)
Discussion started by: mkalase
3 Replies

3. UNIX for Dummies Questions & Answers

cron jobs

Hi, We have a group of hosts using which the cron jobs are submitted... Few days ago i had submitted a cron job in of these hosts, but unfortunately forgot the host name :( Can anyone please help me out in finding this host name from which the cron s submitting the job, i dont want the... (2 Replies)
Discussion started by: bhavanisree
2 Replies

4. UNIX for Advanced & Expert Users

cron jobs...

I need to start a job every friday night at 8:00 P.M , it runs all the day on Sat and Sun....can somebody tell me how to do this...I understand crontab...but haven't used it.........can u write some steps.....how to create a file and call.....I honestly dont know? Plz help.Thanks (2 Replies)
Discussion started by: RubinPat
2 Replies

5. Shell Programming and Scripting

Zipping files older than one month

I have to zip all files older than a month within a directory. I have to archive them using the file extension I have .dat, .csv ,.cnt files within the directory. I used the following command It doesnt work find /path/*.dat -mtime +30 This command doesnot display .dat files older than a... (2 Replies)
Discussion started by: ramky79
2 Replies

6. Linux

cron jobs

I created a php script. I'd like to have it run as a cron job every day at 10:30am. So i added 30 10 * * * /script.php to the cron file Is it possible to run the script on the command line to test it out ? (1 Reply)
Discussion started by: dannyd
1 Replies

7. Solaris

Cron Jobs

whats up, Have some questions about cron jobs, I am fairly new to the unix os. My cronjob sends all output to my mail, does anyone know how to redirect output to come out on the screen??? Thanks...... (6 Replies)
Discussion started by: lewisoco
6 Replies

8. UNIX for Dummies Questions & Answers

Cron Jobs

Where can someone find info on Cron Jobs? Very new to UNIX and the PC I inherited looks to have several of them. Looks like they are some kind of background program that runs automatically at specified times. Would like to delete some of them and know more about them. (6 Replies)
Discussion started by: dereckbc
6 Replies

9. UNIX for Dummies Questions & Answers

cron jobs

I was wondering if itīs possible to cron job not to run on a certian day and time. Iīve got a job that runs everyday at 08:00 but would like it not to run on the 20:th between 08:00 and 10:00 Anyone know if this is possible, and if. How do i do it? regards... dOzY (3 Replies)
Discussion started by: dozy
3 Replies

10. Shell Programming and Scripting

cron jobs

hi folks, I need an explanation on cron jobs with an example (4 Replies)
Discussion started by: vijaysabari
4 Replies
Login or Register to Ask a Question