Shell Programming and Scripting

Hi,

I want to download the latest file published on remote server using SFTP connection.
The job will run at every 15 min hence should download only the latest file. The fileName will be like ABC_DDHHMISS.txt (DD- day, HH-hour, MI - Min, SS-Sec).
So there will be files with same date for previous month, hence while downloading I should avoid old files and extract only latest files.

I tried to use the below script where the TIMESTAMP_FILE will get touched as soon as the script completes so next time when the script will run, it will fetch all new files after the timestamp of TIMESTAMP_FILE.
but when i execute the below script it's unable to find the $TIMESTAMP_FILE as it's available in local server.

Please help me to fix the issue.

Code:

NEW_FILES=`ssh username@hostname find /data/feeds/OUT -type f 
-newer $TIMESTAMP_FILE -name "ABC*"`

Thanks in advance.

Why not rename the files on the server or move them to a backup directory.

Maybe he doesn't have write access to the remote server. Otherwise he could write the time stamp file on the remote server.

Right, We do not have permission to write/move the files of remote server.

Do it in two steps. First get a listing of the files on the remote server. Then examine the list and create a list of files to retrieve.

Try something like this:

Code:

NEW_FILES="ssh username@hostname find /data/feeds/OUT -type f 
-newer `date -d "-15 minutes"` -name "ABC*"

with the command in back ticks.
That will check for files newer than the date - 15 minutes.

This only works with GNU date though. If they are running linux it should be gnu date


EDIT: doesn't look like you can specify a date for -newer :/ I think the 2 step process is going to be your best bet.

Can it work this way -
when the script executes first time, it will fetch all the files published for respecitve date. once received to local drive, when the script executes for next interval, we can read the last file received at first run, and send the same filename as a timestamp file to check on remote server and can retrive the rest of the files published after it.

Also, when I exeute the complete script, it dosen't do anything. It just run and didnot fetch any records.

when I execute only, ssh username@hostname I am getting below message -

'Last login: Fri Mar 26 04:24:37 2010 from ny1gmcollatp1.u
Access to this computer is prohibited unless authorised
Accessing programs or data unrelated to your job is prohibited
If you are not authorised, disconnect now.'

but when i use sftp username@hostname, I am able to connect without any issues.