Argument list too long - Shell error


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Argument list too long - Shell error
# 1  
Old 03-02-2006
Question Argument list too long - Shell error

Trying to tar specific files from a directory causes problems when the number of files is too large.

ls ~/logs | wc -l
5928

In the logs directory - I have 5928 files

If I want to include all files with today's date - I run the following command

tar cf ~/archive/LoadLogs_20060302.tar ~/logs/LoadLog_20060302_*.log

However, I get the following error:

/usr/bin/tar: Argument list too long

Any suggestions on how to correct this would be appreciated.

Thanks,

Rick

Last edited by dad5119; 03-02-2006 at 04:31 PM..
# 2  
Old 03-02-2006
Your tar command doesn't look right. You need something like:
tar cf ~/archive/LoadLogs_20060302.tar ~/logs/LoadLog_20060302_*.log
# 3  
Old 03-03-2006
You can specify an "include file" usually with a -I option. To use, create a directory listing into the include file:

ls -1 ~/logs/LoadLog_20060302_*.log > /tmp/tar.include

Then run the tar command

tar -cf ~/archive/LoadLogs_20060302.tar -I /tmp/tar.include
# 4  
Old 03-03-2006
Argument list too long for multiple commands

Thanks for the suggestions - however when I use an asterisk in my matching criteria (even for the "ls" command) - I get the same error.

I guess the default limit for an argument list is used for ALL commands.
Commands like "ls" "gzip" "tar" "cp" "mv" "cat" - all complain when I try to limit the argument list.

I was hoping there was some shell configuration setting that would increase the allowable size - but I'm beginning to think that it's the programs problem - not the shell environment.

Rick
# 5  
Old 03-03-2006
Quite right, I should have seen that coming. Use the find command to generate the include file:

find ~/logs/ -name "LoadLog_20060302_*.log" > /tmp/tar.include

This isn't AIX by chance is it?
# 6  
Old 03-03-2006
not c shell by any chance?
# 7  
Old 03-03-2006
Not C shell.
Running ksh on Unix and Linux - same results on both.

Thanks
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Unexpected Argument list too long error on later os level

I have a script on RedHat which runs this: ssh -q -t -i $HOME/.ssh/my_key server2 "find ~/toCopy/*data* | xargs sudo mv -f -t /home/files/" I'm getting: sudo: unable to execute /bin/mv: Argument list too long but the reason I use xargs is to avoid this restriction. It used to work... (8 Replies)
Discussion started by: say170
8 Replies

2. Shell Programming and Scripting

Argument list too long

Hi Team, Here's the situation. I have approximately 300000 to 500000 jpg files in /appl/abcd/work_dir mv /appl/abcd/work_dir /appl/abcd/process_dir The above move command will work if the jpg files count is close to 50000 (not sure). If the count is less this mv command holds good. But if... (14 Replies)
Discussion started by: kmanivan82
14 Replies

3. Shell Programming and Scripting

mv : Argument list too long

Hi I am using find command -- find "directory1" -type f | xargs -i mv {} "directory2" to avoid above argument list too long problem. But, issue i am facing is directory1 is having subdirectories due to this i am facing directory traversal problem as i dont want to traverse subdirectories... (9 Replies)
Discussion started by: VSom007
9 Replies

4. Shell Programming and Scripting

grep : Argument list too long

Hi, i am having some trouble with the below command, can some one suggest me the better way to do it. grep -l 'ReturnCode=1' `find $Log -newer /tmp/Failed.tmp -print | xargs ls -ld | egrep SUB | egrep -ve 'MTP' -ve 'ABC' -ve 'DEF' -ve 'JKL' -ve 'XYZ' | awk '{print $9}'` > $Home1 Its... (2 Replies)
Discussion started by: Prateek007
2 Replies

5. Shell Programming and Scripting

Argument list too long!!

Dear Experts, I have a list of 10K files in a directory. I am not able to execute any commands lile ls -lrt, awk, sed, mv, etc........ I wanna execute below command and get the output. How can I achieve it?? Pls help. root# awk -F'|' '$1 == 1' file_20120710* | wc -l /bin/awk: Argument list... (2 Replies)
Discussion started by: Naga06
2 Replies

6. Shell Programming and Scripting

Argument list too long problem

I have a huge set of files (with extension .common) in my directory around 2 million. When I run this script on my Linux with BASH, I get /bin/awk: Argument list too long awk -F'\t' ' NR == FNR { a=NR } NR != FNR { sub(".common", "", FILENAME) print a, FILENAME, $1 } '... (1 Reply)
Discussion started by: shoaibjameel123
1 Replies

7. Shell Programming and Scripting

Argument too long list error

I have a wrote a script which consits of the below line.. Below of this script I'm getting this error "ksh: /usr/bin/ls: arg list too long" The line is log_file_time=`ssh -i $HOME/.ssh/id_rsa -q $i ls -lrt /bp/karthik/test/data/log/$abc*|tail -1|awk '{print $8}'` And $abc alias is as "p |... (1 Reply)
Discussion started by: 22karthikreddy
1 Replies

8. Shell Programming and Scripting

"Argument list too long" error

Hi everyone, I have a problem with my shell script. As a quick overview I need to change a template file 6561 times and copy the file into a new catalogue. Thanks to your forum I have managed to write a script to do so: #!/bin/sh template=$1 for values in {45,165,285}\ {45,165,285}\... (6 Replies)
Discussion started by: mario8eren
6 Replies

9. Shell Programming and Scripting

TAR Files Argument list too long error

Hi, I have a requirement where I need to TAR more than 50K files. Even though I can do TAR successfully on few 100s of files, but whenever Im trying to TAR the entire 50K files, I am getting the error message : Argument List Too Long. Please suggest how can i avoid this error. Im... (2 Replies)
Discussion started by: unx100
2 Replies

10. UNIX for Dummies Questions & Answers

Argument list too long - SSH

Hi I executed the code for file in `ls pdb*.ent` do new_name=`echo $file | sed 's/^pdb//;s/.ent/.txt/'` mv $file $new_name done Its giving error at ' ls pdb*.ent' argument list too long i have around 150000 entries please help Thank you (6 Replies)
Discussion started by: empyrean
6 Replies
Login or Register to Ask a Question