Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Identify log files based on time stamp,zip and then copy..HELP

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Identify log files based on time stamp,zip and then copy..HELP
# 1  
Old 11-17-2010
Identify log files based on time stamp,zip and then copy..HELP
Hi All,
PFB is a requirement. I am new to shell scripting. So plz help. It would be highly appreciated.

1. choose all the log files based on a particular date (files location is '/test/domain')--i.e,we should choose all the files that are modified on 29th November, neither 28th nor 30th
2. zip the files (concatenating the files into a single one is OK)
3. name the zip file as the current date (when the shell script is run)
4. copy the zip file to backup_log folder (location would be '/test/domain/backup_log')
-----------------------------------------------------------------------
example of a log file name is application.log.1, application.log.2 and so on

---------- Post updated at 02:50 PM ---------- Previous update was at 02:48 PM ----------

by the way it is ksh UNIX environ.
# 2  
Old 11-17-2010
here is a ksh script that uses awk and gzip (this will give a GNU zip file not pkzip). Do you have a pkzip program installed on your UNIX environ? If you are on solaris you may have the /usr/local/zip command.

backup_domain.sh
Code:
#/bin/ksh
month=$1
day=$2
cd /test/domain
files=`ls -l . |  awk ' $1 ~ /^[^d]/ && tolower($6$7) ~ /'${month}${day}'/ { print $9 }'`
[ -n "$files" ] && cat $files | gzip -9c > backup_log/`date +%Y_%m_%d`.log.gz

Call it like this

Code:
$ backup_domain.sh nov 29

# 3  
Old 11-17-2010
thanks chubler. actually the script needs to pick up all the log files created on the previous day (for example, if the script is run on 28th, it should pick up log files with timestamp of 27th). And it should be free of any argument so that the script running can be automated.

Also, uaing gzip is ok.

Thanks
Nil
# 4  
Old 11-17-2010
That makes more sense. Do you have GNU date (supports "date --date -1day").

This is also really dependant on the format of ls, I've tested on AIX and my ls output looks like this:
Code:
-rwxr-xr-x    1 chubler users           330 Oct 22 09:01 test
-rw-r--r--    1 chubler users          1301 Oct 13 07:40 test.dat
-rw-r--r--    1 chubler users            38 Oct 29 07:49 test1
drwxr-xr-x    2 chubler users           256 Nov 18 09:29 testdir
-rwxr-xr-x    1 chubler users           400 Oct 29 07:34 testi


If all is OK this should work for GNU date:

Code:
#/bin/ksh
when=`date --date -1day +%b%d`
cd /test/domain
files=`ls -l . | awk ' $1 ~ /^[^d]/ && $6$7 ~ /'${when}'/ { print $9 }'`
[ -n "$files" ] && cat $files | gzip -9c > backup_log/`date +%Y_%m_%d`.log.gz

Otherwise this might do it (Adjust +14 for you timezone, we want to get date reporting the same time but with yesterday's date):

Code:
#/bin/ksh
when=`TZ=GMT+14 date +%b%d`
cd /test/domain
files=`ls -l . |  awk ' $1 ~ /^[^d]/ && $6$7 ~ /'${when}'/ { print $9 }'`
[ -n "$files" ] && cat $files | gzip -9c > backup_log/`date +%Y_%m_%d`.log.gz

Last edited by Chubler_XL; 11-18-2010 at 12:04 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Find and copy .zip file based on today's date

Hi Team, I'm new to unix and i have a requirement to copy or move files from one directory to another based on current date mentioned in the .zip file name. Note that i need to copy only the recent zip file. please help me with the code i tried the code as: #! /usr/bin/sh find... (3 Replies)
Discussion started by: midhun3108
3 Replies

2. Shell Programming and Scripting

Search Files on a given path based on latest time stamp

find /app/data -name "Availability" - Below is the output now i need to filter based on latest modified timestamp. I know 3 is the latest modified time stamp but i tried different options but only filtering docs and not on headnote..Can any one tell me how to do that.. ... (2 Replies)
Discussion started by: vkiyv05
2 Replies

3. Shell Programming and Scripting

Need Time Stamp Range On Log Files

I have created this script #!/bin/sh FILES=/data/log/access_*.log for f in $FILES do echo "Processing $f file" cat $f | awk '{print $1}' | sort | uniq -c | sort -n | tail done It produces this output Processing /data/log/access_abc.log file 114 1.1.1.1 167 2.2.2.2 ... (38 Replies)
Discussion started by: sharingsunshine
38 Replies

4. Programming

How to search a file based on a time stamp backwards 10 seconds

Hi all, I'm after some help with this small issue which i'm struggling to work out a fix for. I have a file that contains records that all have a time stamp for each individual record, i need to search the file for a specific time stamp and then search back 10 seconds to see if the number... (2 Replies)
Discussion started by: sp3arsy
2 Replies

5. Fedora

Move file based time stamp

Hi all, I've already tired to try to solved this problem. Also search in Internet didn't find anything solution I have a directory like this : # pwd /opt/projects/juventini # ls -al | more total 3627460 drwxr-xr-x 2 app apps 12472320 Sep 24 14:59 . drwxr-xr-x 11 app apps 4096 Jun... (8 Replies)
Discussion started by: sunardo
8 Replies

6. Shell Programming and Scripting

how to copy an alertlog file from a particular time stamp?

Hi, Can any one tell me how to copy an alertlog file from a particular time stamp using shell script? Thanks (3 Replies)
Discussion started by: suman_dba1
3 Replies

7. UNIX for Dummies Questions & Answers

How to search for files based on the time stamp

Hi All, I know the timestamp of a file. Now i would like to list all the files in the with the same time stamp in the same file. Any help would be appreciated. Thanks. sunny (1 Reply)
Discussion started by: sunny_03
1 Replies

8. UNIX for Dummies Questions & Answers

Copy all the files with time stamp and remove header,trailer from file

All, I am new to unix and i have the following requirement. I have file(s) landing into input directory with timestamp, first i want to copy all these files into seperate directory then i want to rename these files without timestamp and also remove header,trailer from that file.. Could... (35 Replies)
Discussion started by: ksrams
35 Replies

9. UNIX for Dummies Questions & Answers

Need to delete the files based on the time stamp of the file

Hi Everyone, I want to delete some files in a path based on the time stamp of the file that is i want to delete the file once in a month. Can any one help me on this? Thanks in advance (2 Replies)
Discussion started by: samudha
2 Replies

10. UNIX for Dummies Questions & Answers

How to Zip the files from date Stamp to end date Stamp

Hi, I need to zip the list of files using from date Stamp to end date Stamp, How can I filter and make FromDate_EndDate.gzip? any idea? (1 Reply)
Discussion started by: redlotus72
1 Replies
Login or Register to Ask a Question