Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

unable to do "ls" with more than 90000 files

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers unable to do "ls" with more than 90000 files
# 1  
Old 02-19-2009
unable to do "ls" with more than 90000 files
Hi,

My code is as follwos:

for trg_file in `ls -tr *.trg`
do
<process>

done


The process to be done for each and every file with extension as .trg

When i execute the script I get the follwoing error:
trg_prcs.ksh: line 1: /bin/ls: Argument list too long

How this can be avoide ? Please help me out.
# 2  
Old 02-19-2009
You could use xargs

ls -tr *.trg | xargs processcommand
# 3  
Old 02-20-2009
Quote:
Originally Posted by glisha
You could use xargs

ls -tr *.trg | xargs processcommand
That will still throw an error message from ls. You could try
Code:
find . -prune -name '*.trg'

except if you need the reverse time sort from ls.
# 4  
Old 02-20-2009
Just listing the directory avoids all argument size problems, and you can still filter out the wrong files with grep. I would do:

Code:
ls -tr ./ | grep "\.trg$" | while read trg_file
do
        ....
done

It means launching a few extra processes, but given the sheer number of files involved, the cost may not be overmuch.
# 5  
Old 02-20-2009
Quote:
Originally Posted by risshanth
...
When i execute the script I get the follwoing error:
trg_prcs.ksh: line 1: /bin/ls: Argument list too long

How this can be avoide ? Please help me out.
Simply remove the unneeded ls from your loop:

Code:
for file in *trg
 do 
   # process "$file" 
 done

Or using https://www.unix.com/shell-programmin...d-command.html

Code:
find . \( ! -name . -prune \) -name '*trg'  -print | 
  while read file
    do
        # process "$file"
    done


Tested with about 120000 files in Solaris.

[ If using GNU find use -maxdepth 1 option. ]

--
Just noticed time-reverse ( -tr ) option in ls. If that's definitely needed then use Corona's way.
Last edited by rubin; 02-20-2009 at 12:47 PM.. Reason: added final note
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing... (1 Reply)
Discussion started by: jcdole
1 Replies

2. Shell Programming and Scripting

Delete all log files older than 10 day and whose first string of the first line is "MSH" or "<?xml"

Dear Ladies & Gents, I have a requirement to delete all the log files in /var/log/test directory that are older than 10 days and their first line begin with "MSH" or "<?xml" or "FHS". I've put together the following BASH script, but it's erroring out: for filename in $(find /var/log/test... (2 Replies)
Discussion started by: Hiroshi
2 Replies

3. Red Hat

files having Script which works behind "who" & "w" commands

Dear All, plz print the path of files which have the script of "who" & "w" commands. thnx in advance. (6 Replies)
Discussion started by: saqlain.bashir
6 Replies

4. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

5. Shell Programming and Scripting

"Join" or "Merge" more than 2 files into single output based on common key (column)

Hi All, I have working (Perl) code to combine 2 input files into a single output file using the join function that works to a point, but has the following limitations: 1. I am restrained to 2 input files only. 2. Only the "matched" fields are written out to the "matched" output file and... (1 Reply)
Discussion started by: Katabatic
1 Replies

6. Shell Programming and Scripting

Delete files older than "x" if directory size is greater than "y"

I wrote a script to delete files which are older than "x" days, if the size of the directory is greater than "y" #!/bin/bash du -hs $1 while read SIZE ENTRY do if ; then find $1 -mtime +$2 -exec rm -f {} \; echo "Files older than $2 days deleted" else echo "free Space available"... (4 Replies)
Discussion started by: JamesCarter
4 Replies

7. Shell Programming and Scripting

Unix commands delete all files starting with "X" except "X" itself. HELP!!!!?

im a new student in programming and im stuck on this question so please please HELP ME. thanks. the question is this: enter a command to delete all files that have filenames starting with labtest, except labtest itself (delete all files startign with 'labtest' followed by one or more... (2 Replies)
Discussion started by: soccerball
2 Replies

8. SuSE

VMDB Failure" followed by "Unable to open snapshot file"

keep getting an error when I try to revert to a snapshot: "VMDB Failure" followed by "Unable to open snapshot file" Im using vmware server 1.0.4, host OS is windows xp and guest OS is SLES. Is there anything I can do to recover the snapshot or am I in trouble!?!?! (0 Replies)
Discussion started by: s_linux
0 Replies

9. UNIX for Dummies Questions & Answers

Explain the line "mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`"

Hi Friends, Can any of you explain me about the below line of code? mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'` Im not able to understand, what exactly it is doing :confused: Any help would be useful for me. Lokesha (4 Replies)
Discussion started by: Lokesha
4 Replies
Login or Register to Ask a Question