Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

xargs vs exec with find:

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting xargs vs exec with find:
# 1  
Old 07-14-2012
xargs vs exec with find:
Hi All,

i'm trying to create a tar of all the .txt files i find in my dir . I've used xargs to acheive this but i wanted to do this with exec and looks like it only archives the last file it finds . can some one advice what's wrong here :

Code:
find . -type f -name "*.txt" -print0 | xargs -0 tar -cf txtarchive.tar

$ tar -tvf txtarchive.tar
-rw-r--r-- user/None  123 2012-06-30 17:39 ./employe.txt
-rw-r--r-- user/None   18 2012-07-04 12:59 ./file.txt
-rw-r--r-- user/None    5 2012-07-04 12:59 ./file2.txt
-rw-r--r-- user/None   13 2012-07-14 14:34 ./files.txt
-rw-r--r-- user/None   72 2012-07-12 14:12 ./filesfound.txt
-rw-r--r-- user/None 4938 2012-07-05 09:53 ./sql.txt
-rwxrwxrwx user/None    6 2012-02-15 16:35 ./users.txt

the same action but using exec:

Code:
$ find . -type f -name "*.txt" -exec tar -cvf archivetxt1.tar '{}' \;
./employe.txt
./file.txt
./file2.txt
./files.txt
./filesfound.txt
./sql.txt
./users.txt

$ tar -tvf archivetxt1.tar
-rwxrwxrwx chaitanya/None    6 2012-02-15 16:35 ./users.txt

can some advice whats wrong with my usage of exec here ?

cheers
# 2  
Old 07-14-2012
Quote:
Originally Posted by Irishboy24
Hi All,

i'm trying to create a tar of all the .txt files i find in my dir . I've used xargs to acheive this but i wanted to do this with exec and looks like it only archives the last file it finds . can some one advice what's wrong here :

Code:
find . -type f -name "*.txt" -print0 | xargs -0 tar -cf txtarchive.tar

$ tar -tvf txtarchive.tar
-rw-r--r-- user/None  123 2012-06-30 17:39 ./employe.txt
-rw-r--r-- user/None   18 2012-07-04 12:59 ./file.txt
-rw-r--r-- user/None    5 2012-07-04 12:59 ./file2.txt
-rw-r--r-- user/None   13 2012-07-14 14:34 ./files.txt
-rw-r--r-- user/None   72 2012-07-12 14:12 ./filesfound.txt
-rw-r--r-- user/None 4938 2012-07-05 09:53 ./sql.txt
-rwxrwxrwx user/None    6 2012-02-15 16:35 ./users.txt

the same action but using exec:

Code:
$ find . -type f -name "*.txt" -exec tar -cvf archivetxt1.tar '{}' \;
./employe.txt
./file.txt
./file2.txt
./files.txt
./filesfound.txt
./sql.txt
./users.txt

$ tar -tvf archivetxt1.tar
-rwxrwxrwx chaitanya/None    6 2012-02-15 16:35 ./users.txt

can some advice whats wrong with my usage of exec here ?

cheers
You should use update. Actually you are creating a new tar archive each time overwriting the already existent (if) so only the last found txt will be in tar archive. So it would look like:

Code:
find ./ -type f -name "*.txt" -exec tar -uf myarchives.tar '{}' \;
tar -tvf myarchives.tar

The files are being added in the order they are found by find, not alphabetical or any other. In this case already existing files in tar will remain there even if don't exist anymore in the dir, only existent will be updated.
----------------
To include present and only present remove first the existing tar:
Code:
rm myarchives.tar
find ./ -type f -name "*.txt" -exec tar -uf myarchives.tar '{}' \;
tar -tvf myarchives.tar

-----------------
To always append and don't update existent and don't remove already non existent you can use append instead of update:
Code:
find ./ -type f -name "*.txt" -exec tar -rf myarchives.tar '{}' \;
tar -tvf myarchives.tar

----------------

You can run the exec command at the end of the find search, and redirect all found arguments at once (instead of executing an independent command for each found argument). This is much faster when there are a lot of files, but can cause problems when the list of arguments is too high:

Code:
find ./ -type f -name "*.txt" -exec tar -cvf txtarchive.tar {} + ;

--------------------
With this the maximum number of arguments is solved (thanks to jlliagre for the tip), but the tar must be deleted first (it executes the exec command more than once but much less times than the number of arguments).
Code:
rm myarchives.tar
find . -type f -name "file-*.txt" -exec tar uf myarchives.tar {} +

Last edited by Tribe; 07-14-2012 at 10:46 PM..
This User Gave Thanks to Tribe For This Post:
Irishboy24
# 3  
Old 07-14-2012
cheers tribe, that worked.

I guess xargs builds a file list first before executing the command .

looks like exec is not doing that, do you know of any switch or anything that builds the file list first and then executes the command ?

Thanks
# 4  
Old 07-14-2012
Quote:
Originally Posted by Irishboy24
cheers tribe, that worked.

I guess xargs builds a file list first before executing the command .

looks like exec is not doing that, do you know of any switch or anything that builds the file list first and then executes the command ?

Thanks
So you want to force the create flag?

You can run the exec command at the end of the find search, and redirect all found arguments at once:

Code:
find ./ -type f -name "*.txt" -exec tar -cvf txtarchive.tar {} + ;

Last edited by Tribe; 07-14-2012 at 06:19 PM..
This User Gave Thanks to Tribe For This Post:
Irishboy24
# 5  
Old 07-14-2012
yes, this is what i wanted. it works like a charm. thank you for this.

cheers
# 6  
Old 07-14-2012
Quote:
Originally Posted by Tribe
You can run the exec command at the end of the find search, and redirect all found arguments at once:

Code:
find ./ -type f -name "*.txt" -exec tar -cvf txtarchive.tar {} + ;

Quote:
Originally Posted by Irishboy24
yes, this is what i wanted. it works like a charm. thank you for this.

cheers
Keep in mind that if the size/number of the matching filenames exceeds what can be passed to tar in one invocation, that will silently revert to the original problem, of tar clobbering the archive generated by the previous iteration.

For archiving files found with find, pax or cpio are much more convenient than tar, since they can read the list on stdin.

Regards,
Alister
Last edited by alister; 07-14-2012 at 06:57 PM..
This User Gave Thanks to alister For This Post:
Tribe
# 7  
Old 07-14-2012
Quote:
Originally Posted by alister
Keep in mind that if the size/number of the matching filenames exceeds what can be passed to tar in one invocation, that will silently revert to the original problem, of tar clobbering the archive generated by the previous iteration.

For archiving files found with find, pax or cpio are much more convenient than tar, since they can read the list on stdin.

Regards,
Alister
Exactly, there is a problem if the number of txt existing files exceed the maximum number of arguments that can be handled. For example this will fail on my system:

Code:
for i in {1..50000}; do echo >  file-$i.txt ; done
find ./ -type f -name "file-*.txt" -exec tar -cf txtarchive.tar {} + ;

There are 50000 txt files in dir:
Code:
find ./ -name "file-*.txt" | wc -l
50000

While in the tar there are only 3138 and no error was shown:
Code:
tar -tf txtarchive.tar  | wc -l
3138

-----------
On the other hand, independent exec with find for each argument is terribly slow when the number of arguments is high.

Could you provide examples with pax or cpio?
Last edited by Tribe; 07-14-2012 at 07:12 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Exec and xargs mvoe file to directory

Hello, I am trying to move all the file listed by below command to /tmp/testing directory find ./ -maxdepth 1 -type f -mtime +3 I tried using -exec and xargs - none of the combination is working? Please, help (3 Replies)
Discussion started by: saurabh84g
3 Replies

2. Shell Programming and Scripting

Difference b/w xargs and "-exec" in Find

Hi, What is the difference between the following commands find . -type f -exec grep 'abc' {} \; and find . -type f | xargs grep 'abc' Appreciate your help. (2 Replies)
Discussion started by: bobbygsk
2 Replies

3. Shell Programming and Scripting

find: missing argument to `-exec' while redirecting using find in perl

Hi Friends, Please help me to sort out this problem, I am running this in centos o/s and whenever I run this script I am getting "find: missing argument to `-exec' " but when I run the same code in the command line I didn't find any problem. I am using perl script to run this ... (2 Replies)
Discussion started by: ramkumarselvam
2 Replies

4. Programming

difference bewteen pipe, xargs, and exec

I have read several docs on these on the web and looked at examples. I can't figure out the difference. In some cases you use one or the other or you combine them. can someone help me understand this? (1 Reply)
Discussion started by: guessingo
1 Replies

5. Shell Programming and Scripting

Xargs + Find Help

Guys i want to run a command to list all directories that havn't been modified in over 548 days ( 1.5 yrs ). Id like to run a script to first print what the command finds ( so i get a list of the files pre move ... i have a script set for this : find /Path/Of\ Target/Directory/ -type d -mtime... (4 Replies)
Discussion started by: modulartention
4 Replies

6. Shell Programming and Scripting

find and xargs

hi, i've been trying to figure this weird error but I cannot seem to know why. I am using below find command: find . \( ! -name . -prune \) -type f -mtime +365 -print The above code returns no file because no files are really more then 365 days old. However, when I use xargs, its... (9 Replies)
Discussion started by: The One
9 Replies

7. UNIX for Dummies Questions & Answers

XARGS and FIND together

I am trying to delete files older than 60 days from a folder: find /myfolder/*.dat -mtime +60 -exec rm {} \; ERROR - argument list too long: find I can't just give the folder name, as there are some files that I don't want to delete. So i need to give with the pattern (*.dat). I can... (3 Replies)
Discussion started by: risshanth
3 Replies

8. Shell Programming and Scripting

String substitution on find results inside exec/xargs

What I'm trying to do is perform a copy, well a ditto actually, on the results of a find command, but some inline string substitution needs to happen. So if I run this code find ./ -name "*.tif" I get back these results. .//1234567.tif .//abcdefg.tif Now the action from exec or xargs I... (2 Replies)
Discussion started by: myndcraft
2 Replies

9. Shell Programming and Scripting

MV files with xargs or -exec

Hi I need to move multiple (say 10 files) from one location to another location. My selection would be like this... ls -ltr *.arc | head ---> Need to move top 10 files with single command without iterating in loop. I know we can move files like this with find command but not sure if I can... (4 Replies)
Discussion started by: malaymaru
4 Replies

10. UNIX for Dummies Questions & Answers

Difference between xargs and exec

Hi, I have tried both the options in small dummy scripts, but somehow i can't differentiate between the two. find . -name H* -exec ls -l {} \; find . -name H* | xargs ls -l Both work the ditto way. Any help is appreciated. (19 Replies)
Discussion started by: vibhor_agarwali
19 Replies
Login or Register to Ask a Question