08-19-2009
ok ok we're getting closer
Quote:
#!/bin/bash
function send_ftp
{
ftp -n -i -v > $2 <<EOF
open X X
user X X
epsv4
$1
bye
EOF
}
file_log=log.$$
file_list=tmp.$$
dir_remote=.
#Put the max file that you can copy in one instrucction of mget
max_file=5
#call to generate the list of files in remote host
send_ftp "dir $dir_remote" $file_log
#in my case de first 12lines is the "head" of the log file, besure in you case
#the grep filter is because the lines of ftp begin with number and the files no
sed "1,12 d" $file_log | grep -v ^[0-9] | sed "s/ */#/g"| cut -f9 -d"#" >$file_list
#rm -f $file_log
echo "The file log:$file_log"
#to optimice the ftp comands i make a buffer with the max number of files that we
#can pass to de mget instruction
buffer=""
conta=0
while read file
do
echo "DEBUG:File Name=[$file]"
buffer="$buffer $file"
conta=$(expr $conta + 1)
if [ $max_file -eq $conta ]
then
send_ftp "mget $buffer" "/dev/null"
buffer=""
conta=0
fi
done <$file_list
if [ "$buffer" ]
then
send_ftp "mget $buffer" "/dev/null"
fi
#rm -f $file_list
echo "The file with the list:$file_list"
console gives back something like this:
Quote:
DEBUG:File Name=[XXXX]
DEBUG:File Name=[XXXX]
DEBUG:File Name=[XXXX]
DEBUG:File Name=[XXXX]
DEBUG:File Name=[XXXX]
ftp: No control connection for command.
ftp: No control connection for command.
ftp: No control connection for command.
ftp: No control connection for command.
i needed to add the epsv4 command to get "dir" working
and mget will only work in extended passive mode too
is there any part in your script that got problems with epsv-mode? can't see through this with my low shell script knowledge
i hope there's a solution cause i just failed downloading from this ftp in java
Last edited by macProgger23; 08-19-2009 at 01:09 PM..
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have a shell script where I am trying to ftp some files but I get the error message "EOF unclosed" every time the script reaches the ftp section. Here is how my script is written.
#more code up here
rm -f $object >> $LOG_FILE 2>&1
fi #end of if
done #end of for loop
... (5 Replies)
Discussion started by: matrix1067
5 Replies
2. Shell Programming and Scripting
The following script is used to loop through files in the /tmp directory and transfer those files onto another server.
However, some of the files do not transfer. It is very random when the transferring is done (i.e. one of the files won't transfer then next time, that one will transfer and... (1 Reply)
Discussion started by: RLatham20
1 Replies
3. Shell Programming and Scripting
HI,
I'm downloading one day old files from ftp site.
Below is my script
----------------------------
printf "open $HOST \n" > ftp.cmd
printf "user $USER $PASSWD\n" >> ftp.cmd
printf "bin\n" >> ftp.cmd
#printf "cd /Models/\n" >> ftp.cmd
printf "prompt\n" >> ftp.cmd
printf "for oldfile... (4 Replies)
Discussion started by: shekhar_v4
4 Replies
4. Shell Programming and Scripting
I have a automated FTP script that gets a file using mget. I am using mget because the date will change on the file frequently. The mget works, however if I incorrectly type the file (e.g. if I want to get /dog123 and I enter /dg*) I do not receive and error code from in the FTP session. The... (1 Reply)
Discussion started by: dog123
1 Replies
5. UNIX for Dummies Questions & Answers
Hi calling a batch file with FTP commands from Visual basing with Shell, it basically to copy files then delete them.
The delete seems to work but the copy, it looks like the mget command doesn't have the complete information on where to copy the files.
Here is the batch file content.
Open... (2 Replies)
Discussion started by: Mario120560
2 Replies
6. UNIX for Dummies Questions & Answers
Hi Guys Any Help
I have created a spool file that i need to copy onto another server using FTP in a shell script both servers are linux (3 Replies)
Discussion started by: itai
3 Replies
7. Shell Programming and Scripting
Hi,
i use the below script to send a single file to remote server from linux.
ftp -nvi <<!EOF
open $Host_name
user $USER_ID $PWD
binary
mput $file_name
quit
!EOF (where i... (2 Replies)
Discussion started by: pradebban
2 Replies
8. Shell Programming and Scripting
Hi every one,
I have the requirement to download the files from FTP and move those files to unix box. Once after coping the files, i need to remove the files in FTP.
I'm a newbie in Unix script. Can you please suggest a script for this.. Thanks in advance.. (2 Replies)
Discussion started by: Murali4u
2 Replies
9. UNIX for Advanced & Expert Users
Hi! I am new to unix and this forum as well..
Can someone please help me :
I want to "get/mget" files which are older than 10 minutes from a remote FTP server like "ftp.com".
After getting the files to local unix server say "Prod.com" , i need to delete only those files from ftp.com which... (4 Replies)
Discussion started by: SravsJaya
4 Replies
10. UNIX for Dummies Questions & Answers
Hey All,
first post :rolleyes:
So I am writting a script to pull down files from an ftp that will be called from a bat file on windows. This seems pretty straight forward, and grabs all of the "files" in the cd location, but I am running into some permission issue that will not allow me to... (1 Reply)
Discussion started by: mpatton
1 Replies
AXEL(1) General Commands Manual AXEL(1)
NAME
Axel - A light download accelerator for Linux.
SYNOPSIS
axel [OPTIONS] url1 [url2] [url...]
DESCRIPTION
Axel is a program that downloads a file from a FTP or HTTP server through multiple connection, each connection downloads its own part of
the file.
Unlike most other programs, Axel downloads all the data directly to the destination file, using one single thread. It just saves some time
at the end because the program doesn't have to concatenate all the downloaded parts.
OPTIONS
One argument is required, the URL to the file you want to download. When downloading from FTP, the filename may contain wildcards and the
program will try to resolve the full filename. Multiple URL's can be specified as well and the program will use all those URL's for the
download. Please note that the program does not check whether the files are equal.
Other options:
--max-speed=x, -s x
You can specify a speed (bytes per second) here and Axel will try to keep the average speed around this speed. Useful if you don't
want the program to suck up all of your bandwidth.
--num-connections=x, -n x
You can specify an alternative number of connections here.
--output=x, -o x
Downloaded data will be put in a local file with the same name, unless you specify a different name using this option. You can spec-
ify a directory as well, the program will append the filename.
--search[=x], -S[x]
Axel can do a search for mirrors using the filesearching.com search engine. This search will be done if you use this option. You can
specify how many different mirrors should be used for the download as well.
The search for mirrors can be time-consuming because the program tests every server's speed, and it checks whether the file's still
available.
--no-proxy, -N
Don't use any proxy server to download the file. Not possible when a transparent proxy is active somewhere, of course.
--verbose
If you want to see more status messages, you can use this option. Use it more than once if you want to see more.
--quiet, -q
No output to stdout.
--alternate, -a
This will show an alternate progress indicator. A bar displays the progress and status of the different threads, along with current
speed and an estimate for the remaining download time.
--header=x, -H x
Add an additional HTTP header. This option should be in the form "Header: Value". See RFC 2616 section 4.2 and 14 for details on the
format and standardized headers.
--user-agent=x, -U x
Set the HTTP user agent to use. Some websites serve different content based upon this parameter. The default value will include
"Axel", its version and the platform.
--help, -h
A brief summary of all the options.
--version, -V
Get version information.
NOTE
Long (double dash) options are supported only if your platform knows about the getopt_long call. If it does not (like *BSD), only the short
options can be used.
RETURN VALUE
The program returns 0 when the download was succesful, 1 if something really went wrong and 2 if the download was interrupted. If something
else comes back, it must be a bug..
EXAMPLES
axel ftp://ftp.{be,nl,uk,de}.kernel.org/pub/linux/kernel/v2.4/linux-2.4.17.tar.bz2
This will use the Belgian, Dutch, English and German kernel.org mirrors to download a Linux 2.4.17 kernel image.
axel -S4 ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-2.4.17.tar.bz2
This will do a search for the linux-2.4.17.tar.bz2 file on filesearching.com and it'll use the four (if possible) fastest mirrors for the
download. (Possibly including ftp.kernel.org)
(Of course, the commands are a single line, but they're too long to fit on one line in this page.)
FILES
/etc/axelrc System-wide configuration file. Note that development versions place this file in /usr/local/etc.
~/.axelrc Personal configuration file
These files are not documented in a man-page, but the example file which comes with the program contains enough information, I hope. The
position of the system-wide configuration file might be different.
COPYRIGHT
Axel is Copyright 2001-2002 Wilmer van der Gaast.
BUGS
Please report bugs at https://alioth.debian.org/tracker/?group_id=100070&atid=413085.
AUTHORS
Wilmer van der Gaast. <wilmer@gaast.net>
AXEL(1)