Arg list too long error while performing tar and zip operation


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Arg list too long error while performing tar and zip operation
# 1  
Old 02-28-2019
Arg list too long error while performing tar and zip operation

hi all
i am trying to tar and then zip files present dir by using the below command

Code:
tar -cvf ${abc}/xyz_backup_date_`date +%d%m%y%H%M%S`.tar  xyz*

when the files are in less number the above command executes perfectly but when there are large number of files i am getting "arg list too long" error

please suggest on this error




Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 02-28-2019 at 01:25 PM.. Reason: Added CODE tags.
# 2  
Old 02-28-2019
If your tar supports -T / --files-from:

Code:
ls xyz* | tar --files-from=- --create --file ${abc}/backup_file.tar

...or short...

Code:
ls xyz* | tar -cT - -f ${abc}/backup_file.tar

See: Linux Manpages Online - man.cx manual pages
# 3  
Old 02-28-2019
My tar don't support -T
# 4  
Old 02-28-2019
Check your local tar man-page if tar supports to read file names from stdin.
Another option is cpio - a similar tool - which I'm quite sure it supports reading file names from stdin on your system too, even if i don't know the name and version yet.

If you tell us about specific versions information about your os and your tar command, somebody here may give more information.

Last edited by stomp; 02-28-2019 at 02:32 PM..
# 5  
Old 02-28-2019
Quote:
Originally Posted by stomp
If your tar supports -T / --files-from:

Code:
ls xyz* | tar --files-from=- --create --file ${abc}/backup_file.tar

...or short...

Code:
ls xyz* | tar -cT - -f ${abc}/backup_file.tar

See: Linux Manpages Online - man.cx manual pages
Cramming too many arguments into ls doesn't stop them from being too many arguments. When this happens you have to give up shell globbing.

Code:
rm -f backup_file.tar
ls | grep "^xyz" | xargs tar -rcf backup_file.tar
gzip backup_file.tar

xargs will run tar several times, as many as needed, to append every file to the tarball. (This is why there's an rm -- so that if you run it twice, it will start over instead of making the existing tarball bigger.)

This is less efficient than -T or cpio, especially in that it can't compress the tarball until after it's finished. But if those aren't options, this is how you can do it.
# 6  
Old 02-28-2019
Quote:
ls xyz* | tar -cT - -f ${abc}/backup_file.tar
Um. What did I smoke to write such nonsense? Sorry.

Last edited by stomp; 03-01-2019 at 08:12 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Arg list too long

Hello All, I am trying to find a file name with .sh exention from a list of .dat files inside a directory. find /app/folder1/* -name '*.dat'| xargs grep '.sh' ksh: /usr/local/bin/find: arg list too long Please help me finding the command. Thanks (3 Replies)
Discussion started by: tkhan9
3 Replies

2. Shell Programming and Scripting

arg list too long error

Hello, I'm trying to search through 30,000 files in 1 directory, and am getting the "arg list too long" error. I've searched this forum and have been playing around with xargs and can't get that to work either. I'm using ksh on Solaris. Here's my original code: nawk "/Nov 21/{_=2}_&&_--"... (14 Replies)
Discussion started by: Kristin_in_CO
14 Replies

3. Shell Programming and Scripting

ls command - strange error - arg list too long

I am running a shell script which has the following command ls *.pdf | wc -l error: arg list too long Please post your thoughts on this.. (4 Replies)
Discussion started by: techmoris
4 Replies

4. Shell Programming and Scripting

arg list too long

Hi, Help. I have a file that contains a list of users in a file. I want to cat the content of the file and feed it into sed to a preformated report. The error I got is "ksh: /usr/bin/sed: arg list too long" My method below. A=`cat FILE1.txt` B=`echo $A` sed "s#USERLIST#$B#" FILE2 >... (2 Replies)
Discussion started by: Zenwork
2 Replies

5. UNIX for Dummies Questions & Answers

Arg List too Long in SCP

Hey guys. I have a program written in which i am trying to get the files from one remote machine and transferring the files to another remote machine using SCP. It works fine for 50 or 60 files but when the files grows to 250 then i get an error message stating "Arg list too long". #scp -p... (5 Replies)
Discussion started by: chris1234
5 Replies

6. UNIX for Dummies Questions & Answers

arg list too long when trying to tar files

Hi, I am trying to perform this task: tar -cvf tar.newfile ??????.bas I got error "arg list too long". Is ther any way around? I have about 1500 file need to be tar together. Thanks in advance (5 Replies)
Discussion started by: jds3
5 Replies

7. UNIX for Dummies Questions & Answers

arg list too long when mv files?

hello all i need some help because i am a unix/linux dummy...i have the following: DIR1> has 121437 files in it with varying dates going back to early April, a sub dir DIR1/DIR2> has 55835 files in it I need to move all files (T*.*) out of DIR1 into DIR2 that are older than today? Ive been... (2 Replies)
Discussion started by: jamos007
2 Replies

8. UNIX for Dummies Questions & Answers

ls -t arg list too long

echo dirname/filename* | xargs ls -t As a substitute doesn't give the results desired when I exceed the buffer size. I still want the files listed in chronological order, unfortunately xargs releases the names piecemeal...does anyone have any ideas? :( (4 Replies)
Discussion started by: CSU_Ram
4 Replies

9. UNIX for Advanced & Expert Users

arg list too long

Does anyone have a solution for arg list too long error. I have got this from the web but I fail to make any sense out of it Thanks enc (8 Replies)
Discussion started by: encrypted
8 Replies

10. UNIX for Dummies Questions & Answers

arg list too long

I do ls -l ABC*, I get arg list too long message. This will not happen if ABC* has small no of files I believe 4000 files is limit. Any way of avoiding this. I even tried like this for i in `ls -l ABC*` do echo $i done Same problem. Any solution would be great. I am on HP-UX... (5 Replies)
Discussion started by: vingupta
5 Replies
Login or Register to Ask a Question