UNIX for Dummies Questions & Answers

I have to move files (one by one) from one dir to another, in such a way that the oldest file should be moved first followed by the latest file. The source dir (from where I am moving files) may contains a minimum of 20K files at any point of time. I am not able to use "ls -ltr" as it throws error - Arg list is too long.
Kindly help me in solving this.

1) What platform?
2) Why one by one?
3) We have no clue to help you : What is the delta between oldest and newest?

Are you moving just two files, or are you moving 20k files?

Are you moving files in the unix sense or copying?


You cannot get the error message "Arg list is too long" by typing "ls -ltr" . What is in your script?

Hi Vbe & methyl,

The platform is UNIX Solaris. I will be keep on receiving files from another server to the source dir. I want to move files one by one to the destination dir becoz, there is a daemon application which will be always pooling this destination dir for any files and if a file is found, it starts processing it. This application is taking more time to start when the no: of files in its pooling dir (destin dir) is more. So, in my script I check if the no: of files in the destin dir is < 50 and if so, I have to start moving files (the file with oldest timestamp first, and so on, and file with latest timestamp at last).
At present, I use the following code:

Code:

while [ 1 ];
do
for i in `ls -t $sourceDir`
do
  if [ `find $DestinDir -name "*.txt" | wc -l` -lt 50 ]
     mv $sourceDir/*.txt $DestinDir
  fi
done
done

But this is failing with the error- Arg list too long @ for i in `ls -t $sourceDir`. Please suggest me ways to handle this.
And the delta between the oldest and newest file may be 2 or 3 days, as in worst case we may be having many backlog files in the sourceDir when the daemon application is kept down for maintenance.

---------- Post updated at 07:38 PM ---------- Previous update was at 07:35 PM ----------

small mistake in the code....

mv $sourceDir/$i $DestinDir

Can you do a:

Code:

ls -ltr >$HOME/list_2mv.txt

?
As for It depends on how many... you could use xargs

I'd love to know where this "for i in open-ended-list" construct comes from.

Anyway, we can turn the script from a "for" loop to a "while" loop.
I'd suggest inserting some "sleep" statements in this script because it will loop at a ridiculous rate.
As an idea, try getting out of the inner loop with a "break" command when the destination directory has reached the limit. If you don't there is a good chance that you will process your files list out of order.
There is no code to deal with embryonic files. I hope that they arrive in $sourceDir under a name other than *.txt, then get renamed?

Code:

while true
do
    ls -t "${sourceDir}"/*.txt 2>/dev/null | while read filename
    do
        if [ `find "${DestinDir}" -name "*.txt" | wc -l` -lt 50 ]
        then
            if [ -f "${sourceDir}/${filename}" ]
            then
                mv "${sourceDir}/${filename}" "${DestinDir}"
            fi
        else
            # Stop processing the list if the destination directory is full
            break
        fi
        sleep 1
    done
    sleep 30
done

Something like

Code:

while :
do
let CurrentNum=`find "$DestinDir" -name "*.txt" | wc -l`
while [ $CurrentNum -lt 50 ]
do
{ read filename
mv $filename $DestinDir/${filename##*/}
let CurrentNum++
done } < `ls -rt | egrep -e "txt$"`
done

I have just some doubt about
- where the read should be placed
- where { } should be added
- where the < `ls -rt | egrep -e "txt$"` should be placed

Maybe the read should be moved at the while condition level..
while [ $CurrentNum -lt 50 ] && read filename
...
done < `ls -rt | egrep -e "txt$"`

if you have many files, the wildcard expansion may failed with

Code:

ls -t "${sourceDir}"/*.txt

so i went for a

Code:

`ls -rt | egrep -e "txt$"`

instead.