UNIX for Dummies Questions & Answers

Hello,

Before I do a GET remote file, I need to ensure the remote file is a complete file i.e. whatever process is saving the file to the remote folder should complete the transfer before I go GET it through my script.

So I'm thinking I need to poll the remote file every minute or so to check the file size and if between 2 polls if the file size is same then go ahead and GET the file.

I'm struggling as as this polling needs to be done on the remote folder and
!sleep command does not work in sftp . Any help really appreciated.

Thanks - Neel

If you're using Linux, then check out inotify.
See this: Filesystem Monitoring ?(Tom's Linux Tip's)?

Actually I'm on AIX. Here's a general algorithm on what i'm trying to do.

first_time = true
while /usr/bin/true
do
if first_time ()
ftp get file,
name it as file_1
first_time=false
sleep 100
else
get file, name it as file_2
file_2_size =ls -l | tr -s " " | cut -d" " -f5
file_1_size =ls -l | tr -s " " | cut -d" " -f5
if file_2_size == file_1_size
then
exit loop
else (if file_2.size > file_1.size then
copy file_1 = file_2
sleep 100
fi
fi
done

I guess I need this transalated to unix format. Thanks for the help.

-Neel

---------- Post updated at 12:15 PM ---------- Previous update was at 12:11 PM ----------

Actually this is a second option, if somehow I can do the sleep on the remote server and check the file sizes (before and after) then that should be better than downloading the files to the client and doing the compare.

Thanks -Neel

How does your polling script that check the file on one server know how to tell the ftp script on another server that the file is ok for transfer?

If you are using control-m it's possible, you can have script in Beijing run the file check and when it completes have it pass the control to another ftp script in Malaysia and trigger it. But for normal shell script how you going to do that?

A better solution will be to have 1 script instead of 2. 1 script on the server with the file. The script will do checking then after checking complete the same script will ftp using PUT instead, that way you have better control over the checking.

rsync - if you can install it - will handle all of those issues. It is designed to copy files using ssh transport (like sftp) and keep the transferred files in sync with the current version of the remote file.