UNIX for Dummies Questions & Answers

I have a requirement in which I shud use ls -lrt instead of ls command because when we use ls command we get an error..the script part where i get error is given below

for SAPRESPONSEFILES in `ls $SAPRESPONSEGOFILE | sed "s/go/dat/g"`

basically the script processes the files of format sd63****.dat and sd63****.go files

SAPRESPONSEGOFILE may have types as sd630000 to sd639999

we have a logic built in script which checks for the sequence number last processed.here after processing sd639998 file the files sd639999 ,sd0000 sd 0001 startb to come.Since while lsiting using "ls"
the sd630000 comes first before sd639999 the script fails due to sequence number issue.we tried to use ls -lrt but our team mate says it will have some performance issues.

COULD ANYONE PLEASE HELP ME ON THIS

You probably don't mean:

(ls hyphen-ell-arr-tee)

Try:

Code:

ls -1rt

(ls hyphen-one-arr-tee)


There is no difference in performance between "ls" with or without the "-1rt" because both perform a sort.

How many files do you have in the directory? The "for" construct may break with large numbers of files and may be better with "while".

If I'm not mistake the "sed" will cause ".go" and ".dat" filenames to all become ".dat", thereby creating identical pairs. Is this what you intended, or are you trying to confine processing to just ".go" and ".dat" files?

the prob is not with processing .dat files
files will be coming into the input directory continuously of the format sd63****.dat.the prob is after processing sd639998.dat the file sd9999.dat,sd0000.dat,sd0001.dat .... start to come.while the script list the files after processing sd639998.dat, the sd630000.dat comes before sd639999.dat since we user ls command and then comes the error since the sequence sd639999.dat is missed if we use ls -lrt command it will be fine..but our team mate says there were performance issue when they uses ls -lrt command...

please help me out

The -r flag just reverses the sort, it will still have the skip. Since they're being added continuously, I'm assuming they have sequential timestamps, so if you are using GNU ls, try the --sort=time option, which will sort by time instead of filename.

The performance issue is that, if there are thousands of files -- which there probably are -- it crams them all into one shell variable before continuing. This is slow and may even truncate the list if it is too large. Try something like this instead, piping it:

Code:

ls --sort=time $SAPRESPONSEGOFILE | sed "s/go/dat/g" |
while read SAPRESPONSEFILES
do
  echo "File is ${SAPRESPONSEFILES}"
done

If the performance is too slow you could use shell string manipulation to replace go with dat and leave out sed.

If there are still performance issues with ls, then your directory is too huge, the problem is not ls in particular but rather the rapid statting and sorting of thousands of files in general.

thanks a lot corona...

but why do performance issue occur when we use "ls -lrt" but not when using "ls" as my team mate say.....

[edit] We cross-edited there.

The ls -l performance issue might be because it prints extended information, if your program can't account for that. We haven't seen your program -- we don't know. We thought maybe it could, but if it can't:

Code:

$ ls crc32.c
crc32.c
$ ls -l crc32.c
-rw-r--r-- 1 tyler users 1634 Jul 15 10:04 crc32.c
$

...which would cause your code to theoretically check for files "-rw-r--r--", "1", "tyler", "users", "1634", "Jul", "15", and "10:04" in addition to "crc32.c" Its a good idea to check the manual page to see what the flags actually mean instead of blindly adding them.

Try just ls -t or ls --sort=time (they are equivalent) and see if that gets you the sorting order you want. If its backwards, ls -tr reverses it.

hi that is working...i dont have GNU...

so is used ls -lrt with ur logic..its working....
do u mean to say it will give better perfromance ...

thanks a lot again