Shell Programming and Scripting

Hello everybody

I have wriiten a script that, processes a file, only after its found in a particular folder, the files come in via ftp, everyday at different times, morning, afternoon and evening. The script is run every 5 minutes(0,5,10,15,20...55 minutes) of every hour.

The problem i am facing is these files are quite big, and sometimes, these files come in a few seconds before the script is invoked, by that time, the file transfer is still in progress, but since the script has found the file, it runs the rest of the script on it, without making sure that the file transfer is complete.
resulting in the failure of the job.
I am very new to unix, and I was wondering if theres a way to make sure that the file transfer is complete, or a way to make the script wait until the original file is transferred in full.

Thanks,

use "rsync" for incremental backup and transfer, this should reduce the number of attempts for transferring this file, or, in order to make sure that the files are transferred, put a sleep 100 or 200 secs and give them some time before proceeding with the rest of the code.

Interesting !


File which is in progress can be identified through lsof command,

which would list files opened and the associated process that had opened it.

Consider this example,

#include <stdio.h>

Code:

int main()
{
        while(1)
        {
          fprintf(stdout, "dummy\n");
        }
return 0;
}

When this run as,

Code:

./a.out > dummy

Would be displayed with lsof as,

Code:

lsof dummy
COMMAND   PID    USER   FD   TYPE DEVICE      SIZE    NODE NAME
<Information below>

Since the script is well aware of which it is going to process,
lsof can be done on that script!

When using FTP transfer the file with a "wrong" name. When the transfer is done you rename the file to it's correct name in the same FTP session.

Thank yourl, for the excellent suggestions, now i have 4 great solutions to work around with.

smart suggestion

In my previous project we used to use "lsof" to detect the same thing - as suggested by matrix above.