Shell Programming and Scripting

Hi All,

I have written a script to FTP files from local server to remote server. When i try it for few number of files the scripts runs successfully. But the same script when i run for 200-300 files it gives me performanace issue by aborting the connection.

Please help me out to improve the performance. Script is working fine with respect to functionality.

Thanks.

This is from the wu-ftpd man page .

Code:

The ftp server will timeout an inactive session  after  15
       minutes.   If  the  -t option is specified, the inactivity
       timeout period will be set to timeout seconds.   A  client
       may  also  request a different timeout period; the maximum
       period allowed may be set to timeout seconds with  the  -T
       option.  The default limit is 2 hours.

You may be using a different ftpd, but overall, it might be similar. You just might be hitting the default max timout limit.

Try running a loop to send ten or twenty files at a time to see if it still times out

Hi blowtorch,
I am using the following line in my script.
ftp -ivT -n $HOST_NAME << ENDSCRIPT >> $LOGFILE

When i used it it gave me"invalid option -T"

Trying in a loop will be quite complicated as per as the filenaming conventions and other things are concerned. If you could please suggest me the way to improve performance i will be grateful to you.

Few things : how big are these 200 files ? What is the server timeout ? As blowtorch suggested, in his version the default time out is 15 minutes, in my version of ftpd the default is 2 hours. Check the server timeout on the remote side, if you can. Also, turn on the -v -d flags for verbose and debug, eventually later look at the error codes. What are the operating systems involved, which clients and servers are used ?

Hello All,

Apologies if i come to wrong thread.
Request any one of Unix/Linux masters to clarify on the below.

How far it is feasible to open a new ftp connection for transferring each file when there are multiple files to be sent. I have developed shell script to send all files at single stretch but some how it doesnt suit to the requirement as well. I was asked to do each file transfer differently i.e one file per one ftp.

will it effect the performance of the system? Please throw some light on this.

Thanks,
RSC1985

The answer depend on the case. What's the files size ? In most of the cases spawning more FTP client instances will affect the performance, but if you have, let's say 10 files X 5 MB, it's more wiser to fire the transfers simultaneously, thus using the capabilities of the available TCP resources. Otherwise, you'll be sending all those 10 files sequentially, and the total time for transfers will be ~10 times slower. But once again, you need to go more into details, before we can recommend one approach over other.

For my experience, i am using Net::FTP, for more than 100 files in remote machine, size from 5m to 80m, and file transfered in fork, did not find any problem so far.