Shell Programming and Scripting

How do files appear in the directory, are they copied or moved/renamed.

If copied, then how are you going to avoid copying a half copied file?

Somethine like this?

find . -type f -newer afterme.txt | cpio -dumv <new directory>
touch -t afterme.txt

They're moved from a server to another server. It's the second server that I need to copy the files from.

Touch the file once manually. Then after you have found the list of files touch it again.

manually:

Code:

touch my_touch_file

Then in the script:

Code:

find <dir> -newer my_touch_file > listoffiles.txt
touch my_touch_file
cpio -pdumv /somewhere/else < listoffiles.txt

or do it in the background:

Code:

find <dir> -newer my_touch_file | cpio -pdumv /somewhere/else &
touch my_touch_file
wait

Save the list fist, and then feed that into cpio, otherwise you could get a file that will arrive while you are doing the cpio and you'll never retrieve it.

Porter has a point which I was also wondering about, how can you be sure you won't get a partial? If it is longer term storage it doesn't matter since you will get them next time around, but if the files are being processed then you could end up processing an incomplete file.

Not if the following happens

1. start copying the file into the directory locally

2. find picks up the file and starts copying to remote server

3. copy to remote server finishes (that was started in 2)

4. copying of the file locally finishes (that was started in 1)

5. touch occurs

so the file would be older the than the timestamp file and hence not get picked up next time.

Possible but very highly unlikely if the second option above is followed, or an extension of using two touch files on a round robin and touching before find could be used that would definitely not fail.

Either way it is simpler and more efficient than comparing directory listing.

Thanks for the help. I'll give the suggestions a try.