help with tar & zip only last months(say,Sep) files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting help with tar & zip only last months(say,Sep) files
# 1  
Old 10-02-2012
help with tar & zip only last months(say,Sep) files

Need to
1. archive all the files in a directory from the previous month into a tar/gz file, ignoring all already archived 'tar.gz' files
2. Check created .tar.gz file isnt corrupted and has all the required files in it. and then remove the original files.

I am using a function to get the last month name from the current sys date. After getting just the last months files, I have tried passing complete pathname/filename to tar, just the filename to tar and output of ls -l to tar but none works... I am getting one or other error in the tar step and cant get the syntax right... Please help
Code:
logdir="/home/trial/testing\/"

and I am trying:
Code:
for eachfile in `ls -l  $logdir* | grep -v 'tar.gz$'`; do
        file_month=`echo "$eachfile" | awk -F' ' '{ print $6 }'`
        if [[ "${file_mon}" == "${last_month}" ]]; then
                        echo "$eachfile" >> "$files2process"
        fi
done
        if [[ -f "$files2process" ]] ; then
 archivename="${archivedir}/logarchive_${year}_${last_month}.tar"
       cat "$files2process" | awk -F' ' '{ print $NF }' | sed 's-^-'"${logdir}"'-'| xargs tar -cvf  "$archivename"        #prefixing dir. path to filename b4 tar
                gzip "${archivename}"
                gzip -c "${archivename}.tar.gz" | tar -t >> "filesprocessed.txt"
                      cmp "$files2process" "filesprocessed.txt"
                if [ $? -eq 0 ] ; then 
                    echo "Success" ; cd "$logdir" ; rm `cat $files2process`
                      fi

Error eg.
Quote:
tar: Removing leading `/' from member names
tar: /home/trial/testingtestfile1: Cannot stat: No such file or directory
tar: /home/trial/testingtestfile5: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
gzip: /home/trial/testing/logarchive_2012_Sep.tar: No such file or directory
Quote:
tar: You may not specify more than one `-Acdtrux' or `--test-label' option. Try `tar --help' or `tar --usage' for more information.
---------- Post updated at 05:52 AM ---------- Previous update was at 02:05 AM ----------

If only the below line of code could work , my 'tarring the files' step would be largely resolved, but it gives error. Any idea why ? Smilie
Code:
ls -l "$logdir/" | grep -v "tar.gz" | grep " $last_month " | awk -F' ' '{ print $NF }' | sed 's:^:'"${logdir}"'/:' | xargs tar -cvf "$archive_name"

[ERROR]tar: Removing leading `/' from member names
Cannot stat: No such file or directory[/ERROR]

Last edited by jim mcnamara; 10-03-2012 at 06:21 PM.. Reason: modification of code
# 2  
Old 10-03-2012
A simpler way:
tar up all the files from september in $logdir
create two dummy files with dates
201209010000 beginning of September
201209305959 end of september
Code:
cd $logdir
touch -t 201209010000  /tmp/d1
touch -t 201209010000  /tmp/d2
# back the files we want
tar -cfv /tmp/mytarball.tar $(find . \( -newer /tmp/d1 -a ! -newer /tmp/d2 \) )
if [ $? -ne 0 ] ; then
   echo "tar failed"
   exit 1
fi
# the above traps all tar failures
# do something here to check read the tar file  I do not know what you want
# We could restore the tarball someplace and do a directory compare maybe

gzip /tmp/mytarball.tar 
# /tmp is not good longer term storage

mv /tmp/mytarball.tar.gz /someplace/safe
# delete the files
find . \( -newer /tmp/d1 -a ! -newer /tmp/d2 \) -exec rm {} \;

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How can we Zip multiple files created on the same date into one single zip file.?

Hi all i am very new to shell scripting and need some help from you to learn 1)i have some log files that gets generated on daily basis example: i have abc_2017_01_30_1.log ,2017_01_30_2.log like wise so i want to zip this 4 logs which are created on same date into one zip folder. 2)Post zipping... (1 Reply)
Discussion started by: b.saipriyanka
1 Replies

2. Shell Programming and Scripting

Need Script to ZIP/SAVE & then DELETE Log file & send a mail conformation for any error

ENVIROMENT Linux: RHEL 6.4 Log Path: /usr/iplanet/servers/https-company/logs Log Format: user.log.03-15-2015 I have log4j log rotation enabled rotating files on a daily basis. The rotated logs are NOT compressed & are taking up too much space. I need a script that will run daily that... (1 Reply)
Discussion started by: admin_job_admin
1 Replies

3. Shell Programming and Scripting

How to create zip/gz/tar files for if the files are older than particular days in UNIX or Linux?

I need a script file for backup (zip or tar or gz) of old log files in our unix server (causing the space problem). Could you please help me to create the zip or gz files for each log files in current directory and sub-directories also? I found one command which is to create gz file for the... (4 Replies)
Discussion started by: Mallikgm
4 Replies

4. Red Hat

Need Script to ZIP/SAVE & then DELETE Log file & DELETE ZIPS older than 12 months

ENVIROMENT Linux: Fedora Core release 1 (Yarrow) iPlanet: iPlanet-WebServer-Enterprise/6.0SP1 Log Path: /usr/iplanet/servers/https-company/logs I have iPlanet log rotation enabled rotating files on a daily basis. The rotated logs are NOT compressed & are taking up too much space. I... (7 Replies)
Discussion started by: zachs
7 Replies

5. Shell Programming and Scripting

Script using Sed :Search all patterns & after the last Patter, insert a newLine with Comma Sep Value

I am trying to search the pattern "ARS (11)" and after the LAST pattern, i am trying to open new line and enter text using sed. My Existing Text file is Users.txtpaul, Paul Smith, Stevn Smiley, REQ000001, ARS (11) sam, Sam Martin, Stevn Smiley, REQ000001, ARS (11) mike, Mike Conway, Stevn... (8 Replies)
Discussion started by: evrurs
8 Replies

6. UNIX for Dummies Questions & Answers

TAR and ZIP files

Hi, I need a help with zip and tar. I have no done any scripts before with zip command. What I need to achieve is list files in a directory with a specific name (ID_DATE format- given examples) and then zip (or gunzip which I need to use, I am not sure) with timestamp on the file name and then... (15 Replies)
Discussion started by: Vijay81
15 Replies

7. Shell Programming and Scripting

Searching for a string in .PDF files inside .RAR & .ZIP archives.

Hi, I have got a large number of .PDF files that are archived in .RAR & ZIP files in various directories and I would like to search for strings inside the PDF files. I would think you would need something that can recursively read directories, extract the .RAR/.ZIP file in memory, read the... (3 Replies)
Discussion started by: lewk
3 Replies

8. UNIX for Advanced & Expert Users

How to zip/tar millions of files?

Hi guys, I have an issue processing a large amount of files. I have around 5 million files (some of them are actually directories) in a server. I am unable to find out the exact number of files since it's taking "forever" to finish (See this thread for more on the issue). Anyway, now I... (6 Replies)
Discussion started by: verdepollo
6 Replies

9. Shell Programming and Scripting

To write a shell script which groups files with certain pattern, create a tar and zip

Hi Guru's, I have to write a shell script which groups file names based upon the certain matching string pattern, then creates the Tar file for that particular group of files and then zips the Tar file created for the respective group of files. For example, In the given directory these files... (3 Replies)
Discussion started by: rahu_sg
3 Replies

10. Shell Programming and Scripting

Script for tar and zip based on month & year

Hi Friends, I'm doing on script which finds all the files with time stamp and makes them tar and zip, based on their respective month&year. for instance "mar-2004.tar.zip" will contain all the files which was created/accessed/modified on mar-2004. like this the entire filesystem should be taken... (1 Reply)
Discussion started by: tuxfello
1 Replies
Login or Register to Ask a Question