Check if file is loaded completely and then process the file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Check if file is loaded completely and then process the file
# 1  
Old 06-25-2009
Check if file is loaded completely and then process the file

I need to write a script which checks for files loaded into a folder (files are loaded by ftp from other server) and process the file only if the file is loaded completely.

if the file is not complete in the current run, it must be processed in the next run.

Any suggestions would be welcome
Thanks in advance
# 2  
Old 06-25-2009
How do you know if the file is loaded completely?

Well, if you can mark the last line of the file with something, you could check for that with 'tail -1', and process it if that line exists.

Alternatively, you could get the FTP process to upload a marker file first, then delete that marker file once the main file is uploaded. You could then check for the marker file.

Or, you could watch the size of the file over say a 30 second interval. If the file size has remained static in that time, you could assume that no FTP process is writing to it anymore.

Do any of these suggestions help you?

Last edited by cambridge; 06-29-2009 at 06:35 AM.. Reason: Typo
# 3  
Old 06-25-2009
We had a similar requirement once and we used the third option that cambridge has suggested here... Check the size of the file over a 30 second interval and if the size remains static that means the file has been downloaded successfully...


regards,
Arun.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Pid file and process check

Hello, I am running ubuntu14.04 What I am trying to do is restart a process with a shell when pid is dead. I restored pid nr in a file and check with ps aux | grep -v grep | grep $(cat *.pid)| awk '{ print $2 }' While surfing on google, I have found an answer saying that restoring pid in a... (2 Replies)
Discussion started by: baris35
2 Replies

2. Shell Programming and Scripting

SQLLDR :Data not loaded completely

Hi , I am using below control file LOAD DATA APPEND INTO TABLE LSHADMIN.EG TRAILING NULLCOLS ( STUDY CHAR ) and the text file to load data is CACZ885M2301 When I run below command: sqlldr userid=apps/apps control=/home/appsuser/dataload/ctl_file.ctl... (3 Replies)
Discussion started by: Pratiksha Mehra
3 Replies

3. Solaris

file just loaded does not appear to be executable

Hi When i m trying the boot the system with Primary HDD (c1t0d0s0) -- its solaris 10 I m getting an error "file just loaded does not appear to be executable".....So will anyone share the steps to recover from this stage I also tried to build the corrupted superblock from below command but... (0 Replies)
Discussion started by: taruntan
0 Replies

4. Shell Programming and Scripting

check if some file is in copy process, then transfer it

my user copy large files, and it's take 10min for file to be copied to the server (/tmp/user/ files/), if in the meantime start my scheduled script, then it will copy a part of some file to server1 my idea is to check the file size twice in a short period (1-2 seconds) of time, then compare, if... (5 Replies)
Discussion started by: waso
5 Replies

5. Shell Programming and Scripting

To check wheather a file is downloaded completely or not

I will have two files (which were in .txt format) ftp'ed to a specified directory, from where my ksh picks up each file at a time and starts processing it. So i need to write a script which as soon as find a file should check wheather it is download completely or not. If it is still downloading... (2 Replies)
Discussion started by: vpv0002
2 Replies

6. UNIX for Advanced & Expert Users

Check EOF char in Unix. OR To check file has been received completely from a remote system

Advance Thanks. (1) I would like to know any unix/Linux command to check EOF char in a file. (2) Or Any way I can check a file has been reached completely at machine B from machine A. Note that machine A ftp/scp the file to machine B at unknown time. (5 Replies)
Discussion started by: alexalex1
5 Replies

7. UNIX for Advanced & Expert Users

Using FTP to check whether file is completely FTP... plz find the description below

Hi, We have some clients who will place huge files in to one of the remote server. And the shell script written in our local server to retrieve client files (using FTP) placed on one of the remote server of ours by clients. My question Is there any FTP command/script to check from my local... (1 Reply)
Discussion started by: nmsrao
1 Replies

8. Shell Programming and Scripting

waiting until file loaded

Hello i have DB file load with this command ../SS-Tools/SSdbload -r 10 -il /export/specbackup_db/$b y i should wait until the DB load completed then run the next command $SPECROOT/bin/launchinstdbapp `/usr/bin/hostname` SS n VNM.OUT how this can be done with shell scripting NB: i... (1 Reply)
Discussion started by: mogabr
1 Replies

9. UNIX for Dummies Questions & Answers

How to find File copied completely or else ...

In Unix, I am having one file getting copied to some directory. Which command will help me ensure, that file is not completely copied to the disk? (2 Replies)
Discussion started by: videsh77
2 Replies

10. Filesystems, Disks and Memory

some process writin file - check if complete

Hi folks... some process is writing a file.... as soon as the process starts the file comes there, and its growin.. now i in another script want to ftp the file. i don't know if the file is complete or not. the process which writes the file is some other application and hence can't... (0 Replies)
Discussion started by: sade
0 Replies
Login or Register to Ask a Question

Featured Tech Videos