Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Need help to tar and gzip files per date

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need help to tar and gzip files per date
# 1  
Old 12-12-2011
Need help to tar and gzip files per date
hi,

I have multiple files created per date every day. I want to be able to tar all the files older than 7 days into a single file and then gzip that tarred file and move it to a separate partition. Then delete the actual files.

Eg - "ls -l" shows Dec 8 has 6-8 files created. I want to tar them into a single file Date.tar and gzip them and move it to a separate partition.

The platform I run on does not have perl and its only shell scripting and I am not very good at it.

Please advise.

Thank you, V

Code:
-rw-rw-r--    1 root     root      1116305 Dec  8 00:13 2011-12-08_001334.log
-rw-rw-r--    1 root     root           80 Dec  3 00:58 2011-12-08_001334.logaccount_ptr
-rw-rw-r--    1 root     root        46580 Dec  8 00:13 2011-12-08_001334.loginitial_ptr
-rw-rw-r--    1 root     root        46616 Dec  8 00:13 2011-12-08_001334.logptr
-rw-rw-r--    1 root     root       295855 Dec  8 23:59 2011-12-08_235901.log
-rw-rw-r--    1 root     root           80 Dec  8 00:13 2011-12-08_235901.logaccount_ptr
-rw-rw-r--    1 root     root        12012 Dec  8 23:58 2011-12-08_235901.loginitial_ptr
-rw-rw-r--    1 root     root        12012 Dec  8 23:58 2011-12-08_235901.logptr
-rw-rw-r--    1 root     root       291721 Dec  9 23:59 2011-12-09_235916.log
-rw-rw-r--    1 root     root           80 Dec  8 23:59 2011-12-09_235916.logaccount_ptr
-rw-rw-r--    1 root     root        11892 Dec  9 23:59 2011-12-09_235916.loginitial_ptr
-rw-rw-r--    1 root     root        11892 Dec  9 23:59 2011-12-09_235916.logptr
-rw-rw-r--    1 root     root       280519 Dec 10 23:59 2011-12-10_235900.log
-rw-rw-r--    1 root     root           80 Dec  9 23:59 2011-12-10_235900.logaccount_ptr
-rw-rw-r--    1 root     root        11420 Dec 10 23:58 2011-12-10_235900.loginitial_ptr
-rw-rw-r--    1 root     root        11420 Dec 10 23:58 2011-12-10_235900.logptr
-rw-rw-r--    1 root     root       274462 Dec 11 23:59 2011-12-11_235900.log
-rw-rw-r--    1 root     root           80 Dec 10 23:59 2011-12-11_235900.logaccount_ptr
-rw-rw-r--    1 root     root        11184 Dec 11 23:58 2011-12-11_235900.loginitial_ptr
-rw-rw-r--    1 root     root        11184 Dec 11 23:58 2011-12-11_235900.logptr

---------- Post updated at 11:13 AM ---------- Previous update was at 10:45 AM ----------

I can use the command to list all log files older than 7 days
Code:
find / -mtime +90 -exec ls -l {} \; -name "*.log*"

Now the challenge is to create a tar of the multiple files per day date.tar and then gzip to date.tar.gz
Last edited by methyl; 12-12-2011 at 02:56 PM.. Reason: please use code tags
# 2  
Old 12-12-2011
If I understand you correctly:

Code:
find / -type f -mtime +90 -name "*.log*" | xargs -I{} tar -rf date.tar
gzip date.tar

We use -rf instead of -cf since, if xargs calls tar more than once due to the number of files, we want it to append to the existing .tar, not replace it.

Since you can't append to a compressed .tar, we make it uncompressed then gzip it after.

There's more efficient ways but they only work under GNU/Linux and you didn't say what your system is.
# 3  
Old 12-12-2011
Hi,

If i use the find command below, I still am getting a lot of files older than 7 days created on many different dates.
Code:
find / -mtime +7 -exec ls -l {} \; -name "*.log*"

How can I tar the files that it lists per day and then gzip them. So all files created for eg: on dec 1 should be zipped under dec1.tar and gzip them to dec1.tar.gz and so on for dec 2 etc.

Thanks.
Last edited by Scott; 12-12-2011 at 03:51 PM.. Reason: Code tags
# 4  
Old 12-12-2011
You still haven't told me what your system is, which rules out many solutions I've thought of.

How about this:

Code:
find / -type f -mtime +90 -name "*.log*" |
        awk -F/ '{ split($NF, A, "_"); F[A[1]]=F[A[1]]"|"$0 }
        END     {       for(X in F)     print X F[X];   }' |
while read LINE
do
        # Split in the shell so $1 is the date, $2 ... $N are filenames.
        set  -- $LINE # LINE must NOT be quoted!
        DATE="$1"
        shift # Get rid of $1, so $1...$N-1 are all filenames.  This lets us feed them into tar with "$@"
        echo tar -zcf "$DATE.tar.gz" "$@"
done

Running it on local files here gives me:
Code:
tar -zcf 2011-12-08.tar.gz ./2011-12-08_001334.log ./2011-12-08_001334.logaccount_ptr ./2011-12-08_001334.loginitial_ptr ./2011-12-08_001334.logptr ./2011-12-08_235901.log ./2011-12-08_235901.logaccount_ptr ./2011-12-08_235901.loginitial_ptr ./2011-12-08_235901.logptr
tar -zcf 2011-12-09.tar.gz ./2011-12-09_235916.log ./2011-12-09_235916.logaccount_ptr ./2011-12-09_235916.loginitial_ptr ./2011-12-09_235916.logptr
tar -zcf 2011-12-10.tar.gz ./2011-12-10_235900.log ./2011-12-10_235900.logaccount_ptr ./2011-12-10_235900.loginitial_ptr ./2011-12-10_235900.logptr
tar -zcf 2011-12-11.tar.gz ./2011-12-11_235900.log ./2011-12-11_235900.logaccount_ptr ./2011-12-11_235900.loginitial_ptr ./2011-12-11_235900.logptr

Remove the 'echo' once you've tried a dry run and you're sure it will be creating the archives you want.

find /path/to/files would be far more efficient than find / by the way, since find / trawls your entire system...
# 5  
Old 12-12-2011
Linux mlm-test 2.6.18-92cp #1 SMP Wed Oct 27 15:39:23 IST 2010 i686 i686 i386 GNU/Linux

Its a linux 2.6 SPLAT/Secureplatform from Checkpoint.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Red Hat

Tar with particular date files

I need to find out only Jun 10 created and modified file with tar command ,, I am using Linux red hat x86_64 GNU/Linux. ls -lrt -rw-r--r-- 1 nova nova 2757 Jun 7 11:00 feed.ctl -rwxrwxr-x 1 nova nova 457 Jun 7 11:00 acc_feed.csh -rwxr-xr-x 1 nova nova 473 Jun 7 12:11... (2 Replies)
Discussion started by: balajikalai
2 Replies

2. Shell Programming and Scripting

tar and gzip files

Hi Guys, I am using RHEL5 and Solaris 9 & 10. I want to tar and gzip my files then remove them after a successful tar command... Lets say I have files with extension .arc then I want to tar and gzip these files. After successful tar command I want to remove all these files (i.e .arc). ... (3 Replies)
Discussion started by: Phuti
3 Replies

3. UNIX for Dummies Questions & Answers

Create Tar and GZip files under Unix

Hi All I have 2 tar files and inside a Gzip file (actually its folder) that i got from a Unix user to check the code for him and find some problem. I decompress the Tar file and use 7-ZIP to open the Gzip file and Extract all folders and files. I found the code problem and change it but now when... (1 Reply)
Discussion started by: giladboker
1 Replies

4. Shell Programming and Scripting

tar files by date

Hi, I have a solaris 10, I would like to tar log files by date, or by range of hours into one tar file. All I coud do is by name (very simple...:cool:). (4 Replies)
Discussion started by: pointer
4 Replies

5. Shell Programming and Scripting

tar + gzip + split together

Hi All I need guidance on this requirement . We have a directory structure which has data of approx 100 GB We need to tar the structure then zip it and create different files of not more than 10 GB A separate tar file then a .gz should not be created , on the fly a script is needed... (7 Replies)
Discussion started by: aamir1234
7 Replies

6. UNIX for Dummies Questions & Answers

tar and gzip

Hi, I would like to have a combined gzip and tar that will compress and create multiple output tar.gz files. I want to have multiple files output because i cannot create an archive because there is no more space on my harddisk. I cannot transfer it locally because of slow connection. I want to... (3 Replies)
Discussion started by: tungaw2004
3 Replies

7. UNIX for Advanced & Expert Users

tar/gzip/gz...which one to use?

P0251WLADC.svm_wl1 > /svm_wl1/billing/data/server/archive/ALLEVT $ du -k FEB2006 22050224 FEB2006 As you can see,i have a folder called "FEB2006" which is around 22 GB. i guess zip or compress wont work...( i don know how do we compress a folder) i wished to use ""tar" ( i suppose... (5 Replies)
Discussion started by: abhijeetkul
5 Replies

8. UNIX for Dummies Questions & Answers

TAR and GZIP help

Hi, There are 700 .pdf files in a certain directory on the server and I need to TAR them first and then compress them using GZIP to free up the space. The combined size of the .pdf files is 3gb. However, there is only 1gb of free space on the server. So as you can see when I try to TAR these... (3 Replies)
Discussion started by: VandeMatram
3 Replies

9. UNIX for Dummies Questions & Answers

can i tar and gzip in one liner ?

hello can i combine this 2 commands in one liner command? (1 Reply)
Discussion started by: umen
1 Replies

10. UNIX for Dummies Questions & Answers

Combine tar and gzip together?

Hello I just wandering, instead to doing "tar cvf foo.tar * " and then gzip foo.tar , can't it be combined to one command ? (1 Reply)
Discussion started by: umen
1 Replies
Login or Register to Ask a Question