Arg list too long error while performing tar and zip operation


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
# 1  
Arg list too long error while performing tar and zip operation

hi all
i am trying to tar and then zip files present dir by using the below command

Code:
tar -cvf ${abc}/xyz_backup_date_`date +%d%m%y%H%M%S`.tar  xyz*

when the files are in less number the above command executes perfectly but when there are large number of files i am getting "arg list too long" error

please suggest on this error




Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 02-28-2019 at 02:25 PM.. Reason: Added CODE tags.
# 2  
If your tar supports -T / --files-from:

Code:
ls xyz* | tar --files-from=- --create --file ${abc}/backup_file.tar

...or short...

Code:
ls xyz* | tar -cT - -f ${abc}/backup_file.tar

See: Linux Manpages Online - man.cx manual pages
# 3  
My tar don't support -T
# 4  
Check your local tar man-page if tar supports to read file names from stdin.
Another option is cpio - a similar tool - which I'm quite sure it supports reading file names from stdin on your system too, even if i don't know the name and version yet.

If you tell us about specific versions information about your os and your tar command, somebody here may give more information.

Last edited by stomp; 02-28-2019 at 03:32 PM..
# 5  
Quote:
Originally Posted by stomp
If your tar supports -T / --files-from:

Code:
ls xyz* | tar --files-from=- --create --file ${abc}/backup_file.tar

...or short...

Code:
ls xyz* | tar -cT - -f ${abc}/backup_file.tar

See: Linux Manpages Online - man.cx manual pages
Cramming too many arguments into ls doesn't stop them from being too many arguments. When this happens you have to give up shell globbing.

Code:
rm -f backup_file.tar
ls | grep "^xyz" | xargs tar -rcf backup_file.tar
gzip backup_file.tar

xargs will run tar several times, as many as needed, to append every file to the tarball. (This is why there's an rm -- so that if you run it twice, it will start over instead of making the existing tarball bigger.)

This is less efficient than -T or cpio, especially in that it can't compress the tarball until after it's finished. But if those aren't options, this is how you can do it.
# 6  
Quote:
ls xyz* | tar -cT - -f ${abc}/backup_file.tar
Um. What did I smoke to write such nonsense? Sorry.

Last edited by stomp; 03-01-2019 at 09:12 AM..
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #419
Difficulty: Easy
Few popular modern Web browsers support JavaScript with built-in interpreters.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Arg list too long

Hello All, I am trying to find a file name with .sh exention from a list of .dat files inside a directory. find /app/folder1/* -name '*.dat'| xargs grep '.sh' ksh: /usr/local/bin/find: arg list too long Please help me finding the command. Thanks (3 Replies)
Discussion started by: tkhan9
3 Replies

2. Shell Programming and Scripting

arg list too long error

Hello, I'm trying to search through 30,000 files in 1 directory, and am getting the "arg list too long" error. I've searched this forum and have been playing around with xargs and can't get that to work either. I'm using ksh on Solaris. Here's my original code: nawk "/Nov 21/{_=2}_&&_--"... (14 Replies)
Discussion started by: Kristin_in_CO
14 Replies

3. Shell Programming and Scripting

ls command - strange error - arg list too long

I am running a shell script which has the following command ls *.pdf | wc -l error: arg list too long Please post your thoughts on this.. (4 Replies)
Discussion started by: techmoris
4 Replies

4. Shell Programming and Scripting

arg list too long

Hi, Help. I have a file that contains a list of users in a file. I want to cat the content of the file and feed it into sed to a preformated report. The error I got is "ksh: /usr/bin/sed: arg list too long" My method below. A=`cat FILE1.txt` B=`echo $A` sed "s#USERLIST#$B#" FILE2 >... (2 Replies)
Discussion started by: Zenwork
2 Replies

5. UNIX for Dummies Questions & Answers

arg list too long when trying to tar files

Hi, I am trying to perform this task: tar -cvf tar.newfile ??????.bas I got error "arg list too long". Is ther any way around? I have about 1500 file need to be tar together. Thanks in advance (5 Replies)
Discussion started by: jds3
5 Replies

6. UNIX for Dummies Questions & Answers

arg list too long when mv files?

hello all i need some help because i am a unix/linux dummy...i have the following: DIR1> has 121437 files in it with varying dates going back to early April, a sub dir DIR1/DIR2> has 55835 files in it I need to move all files (T*.*) out of DIR1 into DIR2 that are older than today? Ive been... (2 Replies)
Discussion started by: jamos007
2 Replies

7. UNIX for Dummies Questions & Answers

ls -t arg list too long

echo dirname/filename* | xargs ls -t As a substitute doesn't give the results desired when I exceed the buffer size. I still want the files listed in chronological order, unfortunately xargs releases the names piecemeal...does anyone have any ideas? :( (4 Replies)
Discussion started by: CSU_Ram
4 Replies

8. UNIX for Advanced & Expert Users

arg list too long

Does anyone have a solution for arg list too long error. I have got this from the web but I fail to make any sense out of it Thanks enc (8 Replies)
Discussion started by: encrypted
8 Replies

9. UNIX for Dummies Questions & Answers

zcat --> Arg list too long

Hi all I have more than 1000 files in a folder and when ever i use a "compress" or "zcat" command it give error /bin/zcat: Arg list too long. . any solution for this :o (3 Replies)
Discussion started by: muneebr
3 Replies

10. UNIX for Dummies Questions & Answers

arg list too long

I do ls -l ABC*, I get arg list too long message. This will not happen if ABC* has small no of files I believe 4000 files is limit. Any way of avoiding this. I even tried like this for i in `ls -l ABC*` do echo $i done Same problem. Any solution would be great. I am on HP-UX... (5 Replies)
Discussion started by: vingupta
5 Replies

Featured Tech Videos