The UNIX and Linux Forums  
Hello and Welcome from United States to the UNIX and Linux Forums! Thank You for Visiting and Joining Our Global Community.

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google unix.com



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
error message lo-lp-kl Linux 1 05-14-2008 09:04 AM
Error Message chapmana UNIX for Dummies Questions & Answers 5 11-29-2006 11:41 AM
getting last error message tttttt UNIX for Dummies Questions & Answers 1 07-26-2006 03:07 PM
Finding a specific pattern from thousands of files ???? aarora_98 Shell Programming and Scripting 6 02-17-2006 08:28 AM
Suppres error message when moving files from empty source folder Steven Shell Programming and Scripting 2 11-19-2001 01:25 PM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 09-27-2006
alexcol alexcol is offline
Registered User
  
 

Join Date: Sep 2006
Posts: 31
Unhappy trnsmiting thousands ftp files and get an error message

Im transmiting thousands ftp files to a server, when type the command mput *, an error comes and say. args list to long. set to I. So ihave to transmit them in batch or blocks, but its too sloww. what shoul i do?. i need to do a program, or with a simple command i could solve the problem?

Last edited by alexcol; 09-27-2006 at 03:00 PM..
  #2 (permalink)  
Old 09-27-2006
Corona688 Corona688 is offline
Registered User
  
 

Join Date: Aug 2005
Location: Saskatchewan
Posts: 1,937
"mput *" is expanded in the shell, before it is executed, to "mput file1 file2 file3 ..." There is generally a limit of 32K or so for how long the line can be, which is why you get this error when trying to match * for thousands of files.

I'm not familiar with mput. Is it possible for it to take a list of files instead of arguments on the commandline? You could just do "ls > /tmp/filelist" to make the list.

You can also use xargs to split down that monolilthic list into more manageable batches. Keep the batches large enough and it shouldn't be too much slower. Try this:
Code:
# List files in the current directory, piping the output into xargs
ls ./ |
# For each batch of 100 or less, execute "mput file1 file2 ... filen" where filen is the nth file name in the batch.
    xargs --max-args=100 mput
ls will list the files one per line, xargs will group them together in batches of 100 and call 'mput file1 file2 file3 ... file100' for each batch.
  #3 (permalink)  
Old 09-27-2006
Andrek Andrek is offline
Registered User
  
 

Join Date: Aug 2006
Posts: 122
Since your "mput'ing" you have access to the source files.
What if you used tar to save all the files to a single tar file and them ftp the single tar file over.....
You could create smaller tar files if the size is to big.

Other options for remote transfer are rcp or scp.

Or nfs mount the file system and copy the files directy.
  #4 (permalink)  
Old 09-27-2006
reborg's Avatar
reborg reborg is online now Forum Staff  
Administrator
  
 

Join Date: Mar 2005
Location: Ireland
Posts: 4,211
or do it on the fly with

put "| tar cf - ." filename.tar
Closed Thread

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 09:36 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0