How do I know the file is arrived?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How do I know the file is arrived?
# 1  
Old 12-21-2010
How do I know the file is arrived?

I hope you guys can help me .
My requirement is as below:

Some data files will be ftped by other machine to my machine to the path /home/input
As soon as the file is ftped I need to zip it and move the file to /home/zipped path.

I can use the below command to zip
zip data.zip data.txt

but how do I find out that file is arrived to /home/input location

The file can arrive any time, so how do I keep watching for it all the day? Is there any command?

Thank you very much.
# 2  
Old 12-21-2010
Sender machine can create empty file, right after main file is transfered. You can then monitor if this empty file exists Smilie
# 3  
Old 12-21-2010
unfortunaly the sender will not send any empty file,
do I need to check for file using while (1) ?
# 4  
Old 12-21-2010
Does /home/input directory ingeneral holds any other files ?

if not , you can do a "ls /home/input|wc -l" and can check the count , if count is zero --> it mean no files transfered , if not --> something arrived.

Regards
Ravi
# 5  
Old 12-21-2010
Some FTP-servers are able to run scripts after uploads - check the documentation for your ftp-server.
I would try avoid a script with an endless loop but schedule a script via cron to run every minute.
If available on your system that script can use lsof (or a similar tool) to check for processes that have this file open (= upload in progress).
# 6  
Old 12-21-2010
If the file is quite small, then you probably won't have to worry about a follow up file to flag completion, so using the -f flag for a test should suffice:-
Code:
while [ ! -f $file ]
do
   echo "`date`: File has not arrived"
   sleep 5
done

If the file is larger, you need to confirm that the whole file has arrived, consider using the fuser command to look at what has the file open:-
Code:
while [ ! -f $file ]
do
   echo "`date`: File has not arrived"
   sleep 5
done
while [ `fuser $file 2>/dev/null | wc -c` -gt 0 ]
do
   echo "`date`: File is arriving"
   sleep 5
done
echo "`date`: File has arrived.  Processing will continue."

The condition for the while loop ask for the processes accessing the file. The standard error is usually the file name, so this is ditched to /dev/null. We then count the bytes returned, so if there is anything there, a process has the file open. If it is zero, then there are no processes working on the file.



I hope that this helps, but please write back if I have missed the point.


Another thought:-
If fuser fails, because of too many processes in total on the machine (we have this problem) there is an open source tool called lsof that can still run. Command required would be:
Code:
lsof +ff $file
... which gives no output at all for a closed file, so you get the answer zero for a closed file or not-zero for an open file.


Robin
Liverpool/Blackburn
UK

Last edited by rbatte1; 12-21-2010 at 09:06 AM.. Reason: Another thought
# 7  
Old 12-21-2010
You can try something like this in order to find out if File has arrived :
Code:
while :
do
if [ "$(ls -A /path/to/monitor/)" ]; then
     echo "File has arrived"
fi
done

Login or Register to Ask a Question

Previous Thread | Next Thread

4 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script (sh file) logic to compare contents of one file with another file and output to file

Shell script logic Hi I have 2 input files like with file 1 content as (file1) "BRGTEST-242" a.txt "BRGTEST-240" a.txt "BRGTEST-219" e.txt File 2 contents as fle(2) "BRGTEST-244" a.txt "BRGTEST-244" b.txt "BRGTEST-231" c.txt "BRGTEST-231" d.txt "BRGTEST-221" e.txt I want to get... (22 Replies)
Discussion started by: pottic
22 Replies

2. Shell Programming and Scripting

Compare 2 text file with 1 column in each file and write mismatch data to 3rd file

Hi, I need to compare 2 text files with around 60000 rows and 1 column. I need to compare these and write the mismatch data to 3rd file. File1 - file2 = file3 wc -l file1.txt 58112 wc -l file2.txt 55260 head -5 file1.txt 101214200123 101214700300 101250030067 101214100500... (10 Replies)
Discussion started by: Divya Nochiyil
10 Replies

3. Shell Programming and Scripting

Match list of strings in File A and compare with File B, C and write to a output file in CSV format

Hi Friends, I'm a great fan of this forum... it has helped me tone my skills in shell scripting. I have a challenge here, which I'm sure you guys would help me in achieving... File A has a list of job ids and I need to compare this with the File B (*.log) and File C (extend *.log) and copy... (6 Replies)
Discussion started by: asnandhakumar
6 Replies

4. Shell Programming and Scripting

how to check the file is arrived

Hi I am doing a perl script . I wanted to check if the files have arrived in the target location for current date. the files format will be some thing like this . So for 13-sep-2012 the file will be like this. feedmgr.usfed.t_bills_20120912_20120913173232... (1 Reply)
Discussion started by: ptappeta
1 Replies
Login or Register to Ask a Question