The UNIX and Linux Forums  
Hello and Welcome from United States to the UNIX and Linux Forums! Thank You for Visiting and Joining Our Global Community.

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google unix.com



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
performance issue vishwaraj Shell Programming and Scripting 1 03-03-2008 02:29 AM
T2000 performance issue freezer91 SUN Solaris 2 02-29-2008 10:23 AM
performance issue big123456 UNIX for Advanced & Expert Users 1 08-28-2007 09:53 AM
performance issue rein AIX 1 07-12-2007 02:54 AM
Performance issue shibz UNIX for Advanced & Expert Users 5 12-17-2002 11:12 AM

Reply
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 08-20-2007
Shiv@jad Shiv@jad is offline
Registered User
  
 

Join Date: Nov 2006
Posts: 27
Performance issue with ftp script.

Hi All,

I have written a script to FTP files from local server to remote server. When i try it for few number of files the scripts runs successfully. But the same script when i run for 200-300 files it gives me performanace issue by aborting the connection.

Please help me out to improve the performance. Script is working fine with respect to functionality.

Thanks.
  #2 (permalink)  
Old 08-20-2007
blowtorch's Avatar
blowtorch blowtorch is offline Forum Advisor  
Supporter
  
 

Join Date: Dec 2004
Location: Singapore
Posts: 2,350
This is from the wu-ftpd man page .

Code:
The ftp server will timeout an inactive session  after  15
       minutes.   If  the  -t option is specified, the inactivity
       timeout period will be set to timeout seconds.   A  client
       may  also  request a different timeout period; the maximum
       period allowed may be set to timeout seconds with  the  -T
       option.  The default limit is 2 hours.
You may be using a different ftpd, but overall, it might be similar. You just might be hitting the default max timout limit.

Try running a loop to send ten or twenty files at a time to see if it still times out
  #3 (permalink)  
Old 08-20-2007
Shiv@jad Shiv@jad is offline
Registered User
  
 

Join Date: Nov 2006
Posts: 27
Hi blowtorch,
I am using the following line in my script.
ftp -ivT -n $HOST_NAME << ENDSCRIPT >> $LOGFILE

When i used it it gave me"invalid option -T"

Trying in a loop will be quite complicated as per as the filenaming conventions and other things are concerned. If you could please suggest me the way to improve performance i will be grateful to you.
  #4 (permalink)  
Old 08-20-2007
sysgate's Avatar
sysgate sysgate is offline Forum Advisor  
Unix based
  
 

Join Date: Nov 2006
Location: Bulgaria
Posts: 1,318
Few things : how big are these 200 files ? What is the server timeout ? As blowtorch suggested, in his version the default time out is 15 minutes, in my version of ftpd the default is 2 hours. Check the server timeout on the remote side, if you can. Also, turn on the -v -d flags for verbose and debug, eventually later look at the error codes. What are the operating systems involved, which clients and servers are used ?
  #5 (permalink)  
Old 07-02-2009
RSC1985 RSC1985 is offline
Registered User
  
 

Join Date: Jun 2009
Posts: 25
FTP-Shell Script-Performance issue

Hello All,

Apologies if i come to wrong thread.
Request any one of Unix/Linux masters to clarify on the below.

How far it is feasible to open a new ftp connection for transferring each file when there are multiple files to be sent. I have developed shell script to send all files at single stretch but some how it doesnt suit to the requirement as well. I was asked to do each file transfer differently i.e one file per one ftp.

will it effect the performance of the system? Please throw some light on this.

Thanks,
RSC1985
  #6 (permalink)  
Old 07-03-2009
sysgate's Avatar
sysgate sysgate is offline Forum Advisor  
Unix based
  
 

Join Date: Nov 2006
Location: Bulgaria
Posts: 1,318
The answer depend on the case. What's the files size ? In most of the cases spawning more FTP client instances will affect the performance, but if you have, let's say 10 files X 5 MB, it's more wiser to fire the transfers simultaneously, thus using the capabilities of the available TCP resources. Otherwise, you'll be sending all those 10 files sequentially, and the total time for transfers will be ~10 times slower. But once again, you need to go more into details, before we can recommend one approach over other.
  #7 (permalink)  
Old 07-03-2009
jimmy_y jimmy_y is offline
Registered User
  
 

Join Date: Jun 2009
Posts: 84
For my experience, i am using Net::FTP, for more than 100 files in remote machine, size from 5m to 80m, and file transfered in fork, did not find any problem so far.
Sponsored Links
Reply

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



All times are GMT -4. The time now is 07:22 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language translation by Google.
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0