Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Files count mismatch when used with Tar with find

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Files count mismatch when used with Tar with find
# 1  
Old 11-30-2011
Files count mismatch when used with Tar with find
Hi

I have used the below steps and found some discrepancies
Code:
step 1 :

find ./ -type f -mtime +7 -name "*.00*" | wc -l   =  13519 ( total files )

( the size of this files is appx : 10GB )

step 2: 
find ./ -type f -mtime +7 -name "*.00*" | xargs tar zcvf Archieve_7.tar.gz  

step 3:

du -h Archieve_7.tar.gz : 12 M

step 4:
then i checked the contents using 

tar -tvf Archieve_7.tar.gz = 1806


why the total 13519 files not archieved in my case ?

any help is much appreciated , thanks
# 2  
Old 11-30-2011
The reason you are not getting what you expect is that xargs is re-invoking tar every time the command line fills. cpio could be used, or you could use the "-T" option of tar:
Code:
find ./ -type f -mtime +7 -name "*.00*" | tar -zcv -f Archieve_7.tar.gz -T -

Or if your version of tar does not support this option, you can append files
Code:
tar -cf Archieve_7.tar README
find ./ -type f -mtime +7 -name "*.00*" | xargs tar -rvf Archieve_7.tar
gzip -9 Archieve_7.tar

tar doesn't let you create an empty archive, so you need to start with something.
Hope this helps.
This User Gave Thanks to m.d.ludwig For This Post:
rakeshkumar
# 3  
Old 11-30-2011
Please post the Operating System and version and Shell.

There is much variation in the behaviour of "tar". There is also variation in the behaviour of xargs. In this case I believe that xargs is generating a command line of the maximum allowed length of a command line and executing "tar" every time this happens (which with 13,519 lines could be several times). Thus you only get the last batch of files.

Might need a two-stage process where you copy the files to an archive directory and then create a compressed tar archive of the whole directory.

Beware that many versions of "tar" will not deal with files larger than 2 Gigabytes.
This User Gave Thanks to methyl For This Post:
rakeshkumar
# 4  
Old 11-30-2011
I am running SUSE 11, (sp2) on z/arch , its called z/linux

tar version is 1.26

Currently am not on the machine to try the above steps, will update you Tomm.

I have written a batch script which takes number of days input and archive the data(deletes the files which archived using --remove-file of tar) ,my script also restores the data to the location taking days input

i have tested it with 700 mb and it worked perfect but failed with 10GB ,

is there any tools available which archives and restore data in linux ??
# 5  
Old 11-30-2011
Quote:
Originally Posted by rakeshkumar
i have tested it with 700 mb and it worked perfect but failed with 10GB
This is probably because tar is being broken into multiple calls by xargs like said above. xargs can only feed tar 64 kilobytes worth of arguments at a time. Each time tar gets called, it overwrites the output file with a new one.

Since you have Linux, tar -T will work and you can use the very first solution you were given. tar will read the filenames directly, without any help from xargs.

On other systems you'd need to workaround with tar -r, which appends the file instead of replacing it.
This User Gave Thanks to Corona688 For This Post:
rakeshkumar
# 6  
Old 12-01-2011
thanks all ,

Code:
find ./ -type f -mtime +7 -name "*.00*" | tar -zcv -f Archieve_7.tar.gz -T -
works for me !!!

are there any other tools which are used for archiving in linux apart from TAR? because i see if tar is been interrupted while processing then that resultant archive is unusable so is there any way to avoid this or any other tools better than "tar "

thanks alot for you support !! Smilie

---------- Post updated at 01:10 AM ---------- Previous update was at 12:48 AM ----------

i have observed that slight change in memory usage

after tar , i have checked usage after extracting the tar.gz and its (du folder) :11049124

compared it using
Code:
find ./ -type f -mtime +7 -name "*.00*" -exec du  {} \;| awk '{s+=$1} END {print "Total SIZE: " s}'

it gave a total of 11048212

the difference is 912 bytes , so are there any header added ?? could any one explain why ?
# 7  
Old 12-01-2011
As far as other tools, cpio and zip come to mind. Back in the olden days - before tar had a -T option, I would have suggested using cpio to resolve your issue.

Yes, a tar file has a header - see this wikipedia article for details.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find Syllable count mismatch

Hello, I have written a syllable splitter for Pseudo English and Indic. I have a large database with the following structure Syllables in Pseudo English delimited by |=Syllables in Devanagari delimited by | The tool produces syllables in both scripts. An example is given below: ... (2 Replies)
Discussion started by: gimley
2 Replies

2. Shell Programming and Scripting

awk to output match and mismatch with count using specific fields

In the below awk I am trying output to one file those lines that match between $2,$3,$4 of file1 and file2 with the count in (). I am also trying to output those lines that are missing between $2,$3,$4 of file1 and file2 with the count of in () each. Both input files are tab-delimited, but the... (7 Replies)
Discussion started by: cmccabe
7 Replies

3. Shell Programming and Scripting

Count mismatch in UNIX

Hi, I have a requirement like below. client is sending the .txt filles.In that file we have 10 records but when I execute the below command it is showing 9 records. klena20> wc -l sample_file.txt|awk '{print $1}' It is showing the output as 9 But in a file records are 10. I found... (7 Replies)
Discussion started by: kirankumar
7 Replies

4. Shell Programming and Scripting

Find *.tar files under all subdirectories

Hi there, I'm new to shell scripting... I've a situation like to find *.tar files under all subdirectories in "/home/abcd" and i used the below, find /opt/lhapp ! -name "temp" | more the above works fine.. Now don't need search few direcotries like "/home/abcd/aaaa",... (15 Replies)
Discussion started by: skcvasanth
15 Replies

5. Shell Programming and Scripting

find files older than and containing then tar.

I'm tring to: find files recursively older than x days that contain dat or DAT then tar them I can find the files older than 90 days containing dat with this: find . -mtime +90 -type f -name "*dat*" -exec tar -cvvfp /some/path/some.tar {} \; but how do I do it case insensitive? ... (3 Replies)
Discussion started by: Ikon
3 Replies

6. UNIX for Dummies Questions & Answers

Count number of compressed files in a tar.gz archive

Hi Folks, I have a tar.gz compressed file with me, and I want to know the number of files in the archive without uncompressing it. Please let me know how I can achieve it. Regards RK Veluvali (5 Replies)
Discussion started by: vrk1219
5 Replies

7. UNIX for Dummies Questions & Answers

tar -cvf test.tar `find . -mtime -1 -type f` only tar 1 file

Hi all, 4 files are returned when i issue 'find . -mtime -1 -type f -ls'. ./ora_475244.aud ./ora_671958.aud ./ora_934052.aud ./ora_934050.aud However, when I issued the below command: tar -cvf test.tar `find . -mtime -1 -type f`, the tar file only contains the 1st file -... (2 Replies)
Discussion started by: ahSher
2 Replies

8. Shell Programming and Scripting

To find String mismatch

Hi, I have a doubt when searching files for the existence of a particular key. I have a property file has data with key and value pair like below and i call it as property file.ini here are the contents in File: popertyfile.ini location.property=2 agent.method=begin newkey=23 ... (2 Replies)
Discussion started by: raghu.amilineni
2 Replies

9. Shell Programming and Scripting

comparing two files and find mismatch

hi i have two files and i want to compare both the files and find out mismatch in 3rd file file1 00354|1|0|1|1|0|0|0|1|2 52424|1|0|1|1|0|0|0|1|2 43236|1|0|1|1|0|0|0|1|2 41404|1|0|1|1|0|0|0|1|2 79968|1|0|1|1|0|0|0|1|2 file2 00354|1|0|1|1|0|0|0|1|2 52424|1|0|1|1|0|0|0|0|2... (9 Replies)
Discussion started by: dodasajan
9 Replies

10. UNIX for Dummies Questions & Answers

find a data from several .tar,.gz files

Hi I have a several files with .tar or .gz in a known location. i have files with one of the several files of .tar or .gz text.txt help.pl move.txt how do i write a script to find out those particular files .tar or .gz where that contains the above mentioned files (i.e,... (2 Replies)
Discussion started by: gkrishnag
2 Replies
Login or Register to Ask a Question