Ftp Limit issue


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Ftp Limit issue
# 1  
Old 01-22-2004
Question Ftp Limit issue

I regularly have to upload files from my Unix server to a windows box. However I'm running into and issue ftp seems to have a limit to the amount of file that it can upload in 1 shot.

What I mean is, say I have 10000 files that start with E

If I try to upload all files via mput E* is get "Arguments too long"

This is causing a big issue for me as I have everything automated via some scripts that I wrote.

I was thinking of some how doing a list then having ftp read that list and upload file by file not via mput e*, but I'm not really sure how to pass a list of files to ftp.

I also looked a Perderabo script that he wrote, but it transfers in the wrong direction for me.

I searched for an answer to this but couldn't find anything, if I missed something, please point me in the direction of that thread.

Any help would be great

Thank you.
# 2  
Old 01-22-2004
This sounds like what you want.. at least the question is the same. The answer, however, I did not test, so basically I'm just throwing a suggestion out there.. Smilie Smilie

http://www.landfield.com/wu-ftpd/mai.../Oct/0013.html
Quote:
Sounds like an internal buffer limitation in the FTP client.

I'd write a shell script to make a file and pipe it into ftp. I do this
by-hand when I'm updating a give repeatedly. Consider:

(echo "bin" ; echo "put master.c" ; echo "quit" ) | ftp ftp.example.com

If I have my username and password in .netrc the login is automatic, the piped commands fire off and I get a command prompt when the transfer's complete. If I have a complex problem, or need to see the ftp output, I use expect to script the ftp client.

Last edited by oombera; 01-22-2004 at 12:19 PM..
# 3  
Old 01-22-2004
A quick fix is to go to the 2nd or 3rd character.

mput E01*
mput E02*
mput E03*
etc
# 4  
Old 01-22-2004
Quote:
(echo "bin" ; echo "put master.c" ; echo "quit" ) | ftp ftp.example.com
I understand some of this bin = change to binary mode and quit = quit the ftp session

but what is put master.c is it just upload that single file or does it upload all the files listed in master.c ? to me it looks like it just uploads the single file.

Quote:
A quick fix is to go to the 2nd or 3rd character.

mput E01*
mput E02*
mput E03*
etc
I'm currently doing this but the files arn't always E's it varies day by day, and is creating a lot of extra work for me.

Is there not a way to pipe the output of directory listing to a file and then in turn pipe that to the ftp program so that it will go file by file instead of having to the mput e* or e01* in my script.

Thanks for the help guys.
# 5  
Old 01-22-2004
do a search for "Arguments too long" on this site.

It was explaned why this happens.
its a limitation on the way the mput gets a list of its arguments.

the quick fix is to tar up all the files then just send the tar file to your windows machine.

OR cut your process up to putting 1024 files at a time.

The below code is untested but it should work.

Code:
#!/usr/bin/perl -w

use strict;
use Net::FTP;

my ($host, $user, $pass, $localfs, $remotefs)=( 'your_host', 'ur_username', 'ur_passwd', 'dir_holding_files' 'dir_2_upload_files);
chdir("$localfs") or die "can not change directory /homes($!)";;
my @ftp_files=glob("*");
$ftp = Net::FTP->new("$host", Debug => 0 ) or warn "Cannot connect to $host: $@";
$ftp->login("$user","$pass") or warn "Cannot login ", $ftp->message;
$ftp->binary();
$ftp->cwd("$remotefs");
for (@ftp_files) { $ftp->put("$_") };
$ftp->quit;


Last edited by Optimus_P; 01-22-2004 at 01:42 PM..
# 6  
Old 01-22-2004
Optimus_P, thanks for the code, 1 small issue, my unix server which is SCO OpenServer Enterprise System (ver 5.0.5m) doesn't have perl unfortunitly.
# 7  
Old 01-22-2004
Well. it's a slow day. So
Code:
#! /usr/bin/ksh

stty -echo
print -n Enter Password-
read PASS
print 
stty echo
exec 4>&1

HOST=xxx.yyy.com
USER=name
DESTDIR=junk
SOURCEDIR=~/tmp

cd $SOURCEDIR

ftp -nv >&4 2>&4 |&
print -p open $HOST
print -p user $USER $PASS
print -p cd $DESTDIR

ls | while read filename ; do
        [[ -f $filename ]] && print -p put $filename
done
print -p close
print -p bye
wait
exit 0

I hope you have ksh.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Solaris

Limit FTP user's access to a specific directory

Hi, I have searched "Limit FTP user's access to a specific directory" subject for 3 days. I found proftp and vsftp but i couldn't compile and install. Is there any idea. Please suggest. (6 Replies)
Discussion started by: hamurd
6 Replies

2. Solaris

How to Limit ftp access parameters for specific users?

Dear friends, :) I create new user useradd -g other -d /export/home/sltftp -m -s /bin/bash -c "SLT user account for TMA ftp backup" sltftp now i need do restrict thees chmod delete overwrite rename from this user:(for all the files in the server ,sltftp user can only able to download... (4 Replies)
Discussion started by: darakas
4 Replies

3. Red Hat

FTP issue

Hi everybody, I would like to findout a directory or a file which is located at ftp server by connecting with ftp <ip> by using any other linux utility. I tried with find and locate commands in this manner but didnt work. Please help me. Regards, Mastan (1 Reply)
Discussion started by: mastansaheb
1 Replies

4. UNIX for Advanced & Expert Users

Ftp issue

Hi we are facing an issue on our server our job connects to a target server and pulls the file from a location .But it is taking too much time to do this . Hwne i manully do the ftp it finished in few seconds .but when the jobs is pulling it is taking 30 to 40 min . please see the log... (1 Reply)
Discussion started by: ptappeta
1 Replies

5. Red Hat

FTP issue

Hi, I have installed RHEL 5 on Vmware. I have configured a simple FTP. I am able to transfer and receive files from the vmware-RHEL to my desktop on which vmware is installed and vice-versa. But when i try from other machine in the network it fails. (2 Replies)
Discussion started by: chetansingh23
2 Replies

6. Shell Programming and Scripting

ftp issue

Hi again, I'm using the following shell script via cron to upload the daily log files from my shell to the web server so we can view them online. But somehow logs get corrupted and i don't know what's issue. I would really appreciate for your help please. Shell Script: ftp.sh #!/bin/sh... (3 Replies)
Discussion started by: user`
3 Replies

7. Solaris

Ftp issue please help.

Hi All. I need to FTP a file say abc.dat to client server say xyz.ibm.com location /etc/passwd/ibm with user name and password. I want to make sure my script when FTP the file abc.dat of 1000kb is correctly got transfered with the same contant every time to Clients server how do I confirm... (9 Replies)
Discussion started by: Haque123
9 Replies

8. Red Hat

limit non login id for ftp

Hi All, How can I limit a userid (/bin/false) to have access only to ftp service? On redhat ent 4. Thanks! (1 Reply)
Discussion started by: itik
1 Replies

9. Shell Programming and Scripting

ftp issue

I was changing a bit of the existing script and didnt comment the ftp part in it. So, the ftp step executed, but there was no input file. I got the message like this netout: write returned 0? 250 Transfer completed successfully. What does this mean?. Has the ftp replaced the existing... (1 Reply)
Discussion started by: dnat
1 Replies

10. UNIX for Dummies Questions & Answers

FTP Issue, Help please

folks; when i manually run this script below, it works fine, but when i run it using a scheduler like "autosys" it doesn't work & i get this error: ftp: connect: A remote host refused an attempted connect operation The script: ftp -n XXX.XXX.XXX.XXX <<- eof User Pswd cd xxx/xxxx/xxx... (0 Replies)
Discussion started by: moe2266
0 Replies
Login or Register to Ask a Question