Need of shell script to download data using ftp


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need of shell script to download data using ftp
# 8  
Old 12-30-2012
Quote:
Originally Posted by sanket Bhargava
Hi , I am very new to this site and as a shell script programmer also i am very novice . can you please tell me how its working .

Thanks in advance.
I have updated this based on some of the feedback from Corona688
Code:
#WRITTEN BY MVONA; FREE TO USE AND ABUSE
#THE WORK "WORKS FOR ME" TM
#THIS SCRIPT CONNECTS TO FTP SERVERS AND DOWNLOADS STUFF
#YOU NEED FOUR COLUMN SPACE SEPARATED FILE CONTAINING IN THE FOLLOWING 
#FORMAT:
#<FTP SERVER ADDRESS> <FULL PATH AND FILE NAME OF THE FILE YOU WANT><LOGIN><PASSWORD>
#
#THIS SCRIPT NEEDS TO BE CALLED BY PASSING THE FULL PATH TO THE ABOVE FILE AS ITS FIRST PARAMETER.
##########################################################
#      ERROR HANDLING                                    #
##########################################################
usage()
    {
    
        echo "Proper usage: $0 <full path to server list>"
        exit 1
    }


if [ $# -ne 1 ]; then
    {
        usage
    }
    elif  [ ! -f $1 ]; then
    {
        echo "I see no file $1"
        usage
    }
    elif [ `awk '{print NF; exit}' $1` -ne 4  ]; then
    {
    echo "I see the file $1, but I don't see 4 columns of data in it..."
        usage
        
    }
fi
##############################################################
#        MAIN ROUTINE                                                         #
##############################################################

cat $1 | grep -v '^#' | (while read server file user pass
do
    echo -e "user $user $pass\n binary\n passive\n get $file\n bye" | ftp -niv $server
done)

Note I added the ability to add comments to the server list by placing a grep statement between cat and the ftp statement, which is why I prefer this method over the simple redirect. I also prefer the echo -e statement since it's easier to code a second case structure later if, for instance, you'll need to use passive and active on other ftp servers.

So now an example server list file might look like this:
Code:
#POUND COMMENTS THE LINE
#FORMAT: SERVER FILE USER PASS
ftp.yoursite.com /pub/docs username password


Last edited by MattyV; 12-31-2012 at 03:54 PM.. Reason: Incorporated some feedback from Corona688
# 9  
Old 12-30-2012
That is a useless use of cat and useless use of awk, the read builtin does not need cat's help to read files, does not need awk's help to split tokens, and does not need an entire subshell to do a loop.

The -r is to prevent read from doing things if someone has a backslash in their password.

Also, many systems do not have echo -e, I suggest a here-document instead, which should work anywhere.

Code:
while read -r server file user pass
do
        ftp -niv $server <<EOF
user $user $pass
binary
passive
get $file
bye
EOF
done < $1

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Ubuntu

Bash script for FTP download -Mysql

Hi guys, I recently managed to write up my working script, but now I have a problem. If the file isn't there in the remote server, my actual script jumps it and all ok, but I need something like this: Search file -> if there, then download -> if not, download next file in the list. Any... (7 Replies)
Discussion started by: virtus96
7 Replies

2. Shell Programming and Scripting

Download latest file via ftp server unix through shell script

Hello this is my first post in this forum , I dont want to be unhappy.. I am writing one script but facing difficulty to find the latest file with some new pattern My requirement is 1. The file is coming like "ABCD-23220140303" at FTP server once in a week. 2. script will run on daily... (3 Replies)
Discussion started by: ajju
3 Replies

3. Shell Programming and Scripting

FTP download using perl script

Need assistance I have a script which i can download the files from ftp server using perl . But i want to download multiple files at a time #!/usr/bin/perl -w use Net::FTP; $ftp = Net::FTP->new("ftp.ncdc.noaa.gov"); $ftp->login('username', 'password'); $ftp->cwd("<dir>");... (9 Replies)
Discussion started by: ajayram_arya
9 Replies

4. Shell Programming and Scripting

Download files every one second using ftp script

Our main Server "Srv1" is used to generate text files based on specified criteria and it is also connected to two clients (pc1 and pc2) which are responsible for getting the files from Srv1 as it follows: 1. pc1 ( which represents my UNIX machine ) uses shell script to copy the files from Srv1 2.... (3 Replies)
Discussion started by: arm
3 Replies

5. Shell Programming and Scripting

lftp script to connect to external sftp site and download to internal ftp and then send email

Hi there, I'm new to shell scripting and need some help if possible? I need to create a shell script (.sh) to run as a cron job on an ubuntu linux server to connect to an external sftp sites directory using credentials (which I have) and then download to our internal ftp server and then copy... (3 Replies)
Discussion started by: ghath
3 Replies

6. Shell Programming and Scripting

ftp script doesn't download jpg properly

ftp script doesn't download jpg properly The downloaded files have color splotches Here is the script: ftp -n me@institute.edu <<END_SCRIPT quote user name quote pass password prompt mget *.jpg quit END_SCRIPT exit 0 cd ../ (2 Replies)
Discussion started by: walforum
2 Replies

7. Shell Programming and Scripting

Help on FTP download using UNIX script

Hi guys, I'm new on this forum and on UNIX. Can somebody help in writing a script to download a file from an FTP server and validating if there is a file to download. If there is a file, I would send it to a mail recipient and if not I would generate an error log. Thanks in advance!:D (1 Reply)
Discussion started by: rjay_45
1 Replies

8. Shell Programming and Scripting

Download data from ftp.

Dear all, I am beginner to shell scripting. I have to download more than 1k files from ftp site. link is below ftp://ftp.ncbi.nih.gov/genomes/Bacteria/ Earlier i was using wget url, for download but the problem is some times folder names get changed, i don't know how to move in directories... (4 Replies)
Discussion started by: admax
4 Replies

9. Linux

shell script to download files from a site?

Hey everyone, my wife has purchased a bundle package of a bunch of images from a site, and now has to download each one of them manually. There are about 500 downloads, and it's quite a hassle to browse to each page and download them all individually. I would like to write a shell script to... (2 Replies)
Discussion started by: paqman
2 Replies

10. Shell Programming and Scripting

script for download files from ftp site

I'm new to scripting. I'm trying to write a script to download files from ftp site, the following is the script and the message i get after running the script. no files were downloaded :( Thanks advance! script: #!/usr/bin/ksh DAY=`date --date="-1 days" +%y%m%d` ftp -v -n "ftp.address" <<... (5 Replies)
Discussion started by: tiff-matt
5 Replies
Login or Register to Ask a Question