Visit Our UNIX and Linux User Community


SFTP script - poll every min to check file complete before transfering


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers SFTP script - poll every min to check file complete before transfering
# 1  
Old 08-09-2009
Error SFTP script - poll every min to check file complete before transfering

Hello,

Before I do a GET remote file, I need to ensure the remote file is a complete file i.e. whatever process is saving the file to the remote folder should complete the transfer before I go GET it through my script.

So I'm thinking I need to poll the remote file every minute or so to check the file size and if between 2 polls if the file size is same then go ahead and GET the file.

I'm struggling as as this polling needs to be done on the remote folder and
!sleep command does not work in sftp . Any help really appreciated.

Thanks - Neel
# 2  
Old 08-09-2009
If you're using Linux, then check out inotify.
See this: Filesystem Monitoring ?(Tom's Linux Tip's)?
# 3  
Old 08-09-2009
SFTP script

Actually I'm on AIX. Here's a general algorithm on what i'm trying to do.

first_time = true
while /usr/bin/true
do
if first_time ()
ftp get file,
name it as file_1
first_time=false
sleep 100
else
get file, name it as file_2
file_2_size =ls -l | tr -s " " | cut -d" " -f5
file_1_size =ls -l | tr -s " " | cut -d" " -f5
if file_2_size == file_1_size
then
exit loop
else (if file_2.size > file_1.size then
copy file_1 = file_2
sleep 100
fi
fi
done

I guess I need this transalated to unix format. Thanks for the help.

-Neel

---------- Post updated at 12:15 PM ---------- Previous update was at 12:11 PM ----------

Actually this is a second option, if somehow I can do the sleep on the remote server and check the file sizes (before and after) then that should be better than downloading the files to the client and doing the compare.

Thanks -Neel
# 4  
Old 08-09-2009
How does your polling script that check the file on one server know how to tell the ftp script on another server that the file is ok for transfer?

If you are using control-m it's possible, you can have script in Beijing run the file check and when it completes have it pass the control to another ftp script in Malaysia and trigger it. But for normal shell script how you going to do that?

A better solution will be to have 1 script instead of 2. 1 script on the server with the file. The script will do checking then after checking complete the same script will ftp using PUT instead, that way you have better control over the checking.
# 5  
Old 08-09-2009
rsync - if you can install it - will handle all of those issues. It is designed to copy files using ssh transport (like sftp) and keep the transferred files in sync with the current version of the remote file.
 

Previous Thread | Next Thread
Test Your Knowledge in Computers #668
Difficulty: Medium
IEEE 802.11 is part of the IEEE 802 set of LAN protocols, and specifies the set of media access control (MAC) and physical layer (PHY) protocols for implementing virtual local area network (VLAN) Wi-Fi computer communication in various frequencies.
True or False?

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to check the status of script for every 5 min?

Hi, Can any1 provide the code for my req: I am loading few set of files into database through one unix script. I ll try to describe the process: load_cdr-->main script Source Feeds are A & B. File in A-akm.dat File in B-bkm.dat Now my script runs through cron jobs everyday...and for both... (6 Replies)
Discussion started by: gnnsprapa
6 Replies

2. Shell Programming and Scripting

How to check whether the sftp script is successful??

hi, how can i check whether the sftp connectivity is successful or not?? i am using expect script to connect to sftp.. sftp_script spawn /usr/bin/sftp abc@ftp.xyz.com expect "abc@ftp.xyz.com's password:" send "password\r" expect "sftp>" send "mput *.txt\r" expect "sftp>" send "bye\r"... (8 Replies)
Discussion started by: Little
8 Replies

3. UNIX for Dummies Questions & Answers

Script to check for file size and then sftp

noted down (44 Replies)
Discussion started by: mirwasim
44 Replies

4. Shell Programming and Scripting

FTP using script not working (for transfering file from a remote unix server to windows PC.)

hi, Im using the following code for FTP #!/usr/bin/ksh ftp -v -n "10.29.45.11" << cmd user "mahesva" "mahesva123" get rtl.tar quit cmd Below is the log when i run the above code ********************************** Connected to 10.29.45.11. 220 (vsFTPd 2.0.1) 530 Please login with USER... (20 Replies)
Discussion started by: dll_fpga
20 Replies

5. Shell Programming and Scripting

Transfering files from windows to unix box through sftp

Hello All, we have scenarion where we need to pull the Files from Windows to Unix Box through SFTP protocol. as per our analysis we did install the cygwin package on the Windows Box to have the openSSH package functionality. 1.Will openSSH help us to achieve the functionality what we are... (3 Replies)
Discussion started by: Amey Joshi
3 Replies

6. Shell Programming and Scripting

To check the lists of file created between 30-60 min

Hi I have to write a script, that will find out the lists of files in a particular directory, which are created between 30-60 min intervals. (3 Replies)
Discussion started by: velocitnitin
3 Replies

7. HP-UX

How to Check Whether Core File complete or truncated

hi I have a core file from customer and i am looking for a way to know whether its complete or truncated? I have a procedure to check core file on solaris , is there any way to know whether core is truncated on HP-UX? from khan (1 Reply)
Discussion started by: khan_069
1 Replies

8. AIX

How to Check Whether Core File complete or truncated

Hi I am looking for a way to confirm whether a core File is complete or Truncated Is there any way from Khan (2 Replies)
Discussion started by: khan_069
2 Replies

9. Filesystems, Disks and Memory

some process writin file - check if complete

Hi folks... some process is writing a file.... as soon as the process starts the file comes there, and its growin.. now i in another script want to ftp the file. i don't know if the file is complete or not. the process which writes the file is some other application and hence can't... (0 Replies)
Discussion started by: sade
0 Replies

Featured Tech Videos