UNIX for Advanced & Expert Users

Hi,

I'm after a bit of advice for the best way to collect files from an ftp server via a unix process. The main issue here is the frequency, the job needs to check for files every minute or so between 8am and 8pm and pull them down to the box if there is anything there.

Originally the transfer was going to be done via a VPN tunnel direct to a directory on my machine which my batch job would poll, but life is never simple. Unfortunately, that has now fallen through and the files are now being placed on an ftp site instead and I have to pull the files from there as security will not allow a push from there when a file is deposited on it, which also would have made life easier.

My question is, is there a way of opening a connection to the ftp site and keeping it open to poll for a file for that long, or am I just going to have to open and close connections repeatedly throughout the day?

Is there a better way?

Cheers
Paul.

Writing a little here-script that checks every minute in the time window you said might be a proper solution. Keeping the connection open without doing anything might kick you out anyway, depending on how the ftp server is configured.

Thanks for the reply.

That's sort of what I was thinking, a little process to connect up and check and then if found kick of the main shell. Just concerned with opening and closing a connection that often, after all that would mean 720 connections to the FTP during the space of a day from my process alone, let alone with the external companies dropping files on there. I'm just not overly keen on that. Then again I could just be being a little precious about it

If you have to check it every minute, that is ok, I think. FTP servers usually can stand a lot more of traffic and parallel connections, not sure what hardware, bandwith etc. you are using. I would set it up, let it run and if the admin of the FTP server sees any problems, I guess he will contact you

Hehehe, many thanks for all the input, very much appreciated

I've just started coding a little process that once called will loop through all day check the FTP and then if returns a success kick off the main shell. I'm going to run some intensive testing this afternoon and see if the network admins complain about it or not. If not then I am in business

Many thanks again

Just a thought, could you seen any validity in adding the machine ident, username/password into the .netrc file? That's not going to make a whole heap of difference is it apart from tidy things up a little.

I'd use the .netrc. It makes keeping the password hidden a little easier.

I like expect but the mentioned options are easier. The benefits with expect are that you can build a loggable interface and get a lot of debugging information, notification and control that are impossible otherwise.