How to make sure a file is uploaded in its entirety


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to make sure a file is uploaded in its entirety
# 1  
Old 10-17-2008
How to make sure a file is uploaded in its entirety

Hi, I have a client who uploads files to my FTP server on a regular basis. Instead of checking the server all the time, I wrote a script that e-mail me to let me know that there is a new file and attaches is it in the e-mail (they're all small compressed archives). The potential issue with the following script is that it will move a file to the processed directory even if it's not fully downloaded. I'm scheduling the script to run every hour on the hour but if the client is in the middle of uploading a file when it runs, I may only get a part of it in the e-mail. Is there a way to make sure that the files isn't being modified before moving it?

Code:
sdir="/home/sftp/"
fc=`ls $sdir |wc -l`
fn=`ls -t $sdir |head -1`
if [ "$fc" -ne 0 ];then
        while [ "$fc" -ne 0 ]
                do
                        ts=`date +%Y%m%d_%H%M%S`
                        ( echo "New file attached"
                                uuencode $sdir$fn $fn
                        ) | mailx -s "New file attached" "you@me.com"
                        mv $sdir$fn /home/sftp_processed/[$ts]$fn
                        fn=`ls -t $sdir |head -1`
                        fc=`ls $sdir |wc -l`
                done
else
        echo "No files found"
fi

Thank you! Smilie
# 2  
Old 10-17-2008
lsof will allow you to see if another process has a given file open.
# 3  
Old 10-17-2008
I don't know if it would impose an unacceptable delay into the script, but a sleep might help. Check the filesize, sleep for 30 (or some other reasonable value) seconds, check filesize again, and compare. If the value is changed, rinse and repeat; if not, move the file.
# 4  
Old 10-17-2008
Quote:
Originally Posted by jim mcnamara
lsof will allow you to see if another process has a given file open.
This is exactly what I needed. Thank you!
# 5  
Old 10-17-2008
Quote:
Originally Posted by treesloth
I don't know if it would impose an unacceptable delay into the script, but a sleep might help. Check the filesize, sleep for 30 (or some other reasonable value) seconds, check filesize again, and compare. If the value is changed, rinse and repeat; if not, move the file.
lsof seems like a neater solution. I still might still use sleep to delay processing if lsof finds an open file ...or I'll just skip the file altogether and let it be processed at the next interval.

Thanks for the suggestions!
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

File uploaded via FTP not visible from Command Line

I am using VSFTPD on amazon web services and have a remote service that uploads files to server via FTP. The files comes through fine and when I log in via FTP client I see them. The problem is the files are supposed to be moved once uploaded but the files are not visible via command line thus the... (2 Replies)
Discussion started by: theman0684
2 Replies

2. UNIX for Advanced & Expert Users

i dont want files which are being uploaded 5 to 10 min ago.

i want to move files from A dir to B dir, say eg: mv *memo*.txt /scripts/memo/ but while doing this new files are being uploaded by users, i dont want files which are being uploaded 5 to 10 min ago. other wise even files which are being currently uploaded will get moved to /scripts/memo. (8 Replies)
Discussion started by: rakesh_mumbai
8 Replies

3. Shell Programming and Scripting

i dont want to move files which are being uploaded 5 to 10 min ago

in /abc/jrd/ users are continuously uploading files memo*.txt i have created a memo.sh file in that i have written cd /abc/jrd/ mv memo*.txt /scripts/memo while files are being moved to this /scripts/memo users who are uploading new file are not getting fully copied i this path i want that... (0 Replies)
Discussion started by: rakesh_mumbai
0 Replies

4. Solaris

Multiple private key to be uploaded

I would like to ask if you have a procedure on how to upload multiple private key for multiple users in solaris? I was only able to add one but when I tried to add several key, it fails. example: a. user1: user1.ppk b. user2: user2.ppk Each with different password on the server. Pls advise (6 Replies)
Discussion started by: lhareigh890
6 Replies

5. UNIX and Linux Applications

PAssword protect uploaded files

Hi, Is it possible to make sure/test that all uploaded files to my FTP will be of .RAR format (that's easy) but also password protected? Thanks (0 Replies)
Discussion started by: saariko
0 Replies

6. Programming

makeutility: how to get the make-file name inside of the make-file?

How I can get the current make-file name in a make-file So, if I run make with specified file:make -f target.mak is it possible to have the 'target' inside of the that 'target.mak' from the file name? (2 Replies)
Discussion started by: alex_5161
2 Replies

7. Shell Programming and Scripting

Get Files from ftp which are uploaded recent week

Hi All, Here is a brief scenario for my requirement .. There is a directory in FTP Server, where would files be uploaded on weekly basic. I need to get those files which are uploaded during this week and not the files which are uploaded the previous week and download them to locale... (1 Reply)
Discussion started by: narramadan
1 Replies

8. UNIX for Advanced & Expert Users

automate backing up uploaded files

Hi, We are running FTP Server on UNIX (Solaris 9). Users login and upload (also download) files whenever required. Now, we have to automate the process which makes a copy of every file immediately after it gets uploaded to the FTP server (by any user). I've ruled out rsync,mirror,filesync... (2 Replies)
Discussion started by: prvnrk
2 Replies

9. Post Here to Contact Site Administrators and Moderators

Unable to open uploaded image

Hi, I am unable to view/open the image I uploded in one of the forums. https://www.unix.com/showthread.php?t=37793&page=2&pp=20 Please help. enc. (2 Replies)
Discussion started by: encrypted
2 Replies
Login or Register to Ask a Question