Bash script for FTP download -Mysql


 
Thread Tools Search this Thread
Operating Systems Linux Ubuntu Bash script for FTP download -Mysql
# 1  
Old 11-28-2015
Bash script for FTP download -Mysql

Hi guys,

I recently managed to write up my working script, but now I have a problem.
If the file isn't there in the remote server, my actual script jumps it and all ok, but I need something like this:
Search file -> if there, then download -> if not, download next file in the list.

Any suggestions?
# 2  
Old 11-28-2015
How does your script look like?
Since you want us to help you imrpove it, we'd like to continue from what is there, and not reinvent the wheel.

But either way you could try:
Code:
list="file1 file2"
for this in $list;do
    if [ -f "$this" ]
    then    echo "Downloading: $this"
    else    continue
    fi
done

hth
# 3  
Old 11-28-2015
Here is my actual code

Code:
#!/bin/bash

for i in {20151101..20151127}; do

mkdir -p /tmp/zip;

wget -qq ftp://ftp_user:ftp_pass@ftp.twt.it/TEKNONET/60468/0000060468"$i"N001.Zip -P /tmp/zip;

mkdir -p /tmp/cdr;

unzip -qq /tmp/zip/0000060468"$i"N001.Zip -d /tmp/cdr;


START=`echo $1 | tr -d _`;

TABLE_NAME=M"$i";


echo "Tabella del mese: "$TABLE_NAME;

if [ -f "/var/lib/mysql/twt/"$TABLE_NAME".frm" ] 
    then echo "Tabella esistente: OK"
    
else 
    echo "Tabella non esistente, la creo";
    SQL="create table "$TABLE_NAME"
    (WHSSID int not null,
     Seriale int not null,
     DataOra datetime,
     Chiamante text,
     Chiamato text,
     Prefisso_num_chiamato int,
     Descr_prefisso_chiamato varchar(50),
     Durata_secondi int,
     Costo_conversazione text,
     Chiamata_urb_extraurb int,
     Chiamata_voce_dati int,
     Chiamata_intratwt int,
     Prefisso_carrier_destinazione int);";
    echo $SQL > /tmp/sql;
    mysql -D twt -u root -pp1ngu1n0 < /tmp/sql;
fi

SQL="load data local infile '/tmp/cdr/0000060468"$i"N001.CDR' INTO TABLE $TABLE_NAME 

FIELDS TERMINATED BY ';' LINES TERMINATED BY '\n'";

echo $SQL > /tmp/sql;

mysql -u root -ppassword --local-infile twt < /tmp/sql;

rm -rf /tmp/*

done


Last edited by Don Cragun; 11-28-2015 at 07:51 PM.. Reason: Add CODE tags again!
# 4  
Old 11-28-2015
Please use code tags as required by forum rules

Not sure I understand your problem. Can't you wget all thre relevant and existing files in one go and then only loop through the list to store them in the DB?
# 5  
Old 11-28-2015
I did not understand the tag question.

Btw, how would you change my script in?
# 6  
Old 11-28-2015
man wget:
Quote:
DESCRIPTION
GNU Wget is a free utility for non-interactive download of files from the Web.
Retrieve all the target files by either specifying wildcard chars or using the --input-file option to wget.
Then, loop through the files received loading them into your DB.
# 7  
Old 11-28-2015
So something like that?

Code:
Do
Wget
Done

For x in ()
Do
Load data infile
Done


Last edited by virtus96; 11-29-2015 at 04:16 AM.. Reason: Add CODE tags again!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Curl , download file with user:pass in bash script

Hello, My question is about curl command. (ubuntu14.04) In terminal, I am able to download my mainfile with: curl -u user1:pass1 http://11.22.33.44/******* When I convert it into bash script like this: #!/bin/bash cd /root/scripts computer_ip=11.22.33.44 curl -u $1:$2... (8 Replies)
Discussion started by: baris35
8 Replies

2. Shell Programming and Scripting

Csv download in a bash script

I am attempting to download a url in csv format. When I download this url in a browser excel opens up and automatically populates with comma separated values. When I try to use curl or wget I get nothing or garbage. This on the command line just hangs wget -b... (2 Replies)
Discussion started by: bash_in_my_head
2 Replies

3. Shell Programming and Scripting

Shell script - Download - Mysql replace

Hi, I have a video script. it has embedded Youtube videos. I want replace them downloaded version mp4 videos. this script has a mysql table. I want search "url_flv" field on table a youtube link if has a youtube link I want download it this command. I want extract uniq_id field for file... (2 Replies)
Discussion started by: tara123
2 Replies

4. Shell Programming and Scripting

Download latest file via ftp server unix through shell script

Hello this is my first post in this forum , I dont want to be unhappy.. I am writing one script but facing difficulty to find the latest file with some new pattern My requirement is 1. The file is coming like "ABCD-23220140303" at FTP server once in a week. 2. script will run on daily... (3 Replies)
Discussion started by: ajju
3 Replies

5. Shell Programming and Scripting

FTP download using perl script

Need assistance I have a script which i can download the files from ftp server using perl . But i want to download multiple files at a time #!/usr/bin/perl -w use Net::FTP; $ftp = Net::FTP->new("ftp.ncdc.noaa.gov"); $ftp->login('username', 'password'); $ftp->cwd("<dir>");... (9 Replies)
Discussion started by: ajayram_arya
9 Replies

6. Shell Programming and Scripting

Download files every one second using ftp script

Our main Server "Srv1" is used to generate text files based on specified criteria and it is also connected to two clients (pc1 and pc2) which are responsible for getting the files from Srv1 as it follows: 1. pc1 ( which represents my UNIX machine ) uses shell script to copy the files from Srv1 2.... (3 Replies)
Discussion started by: arm
3 Replies

7. Shell Programming and Scripting

Need of shell script to download data using ftp

Hi ! I am just wondering shell script to download data from ftp... I have text file containing ftp address...looks like this ftp://site...../filename.xyz ftp://site...../filename.xyz ftp://site...../filename.xyz ftp://site...../filename.xyz script has to read ftp address and... (8 Replies)
Discussion started by: nex_asp
8 Replies

8. Shell Programming and Scripting

ftp script doesn't download jpg properly

ftp script doesn't download jpg properly The downloaded files have color splotches Here is the script: ftp -n me@institute.edu <<END_SCRIPT quote user name quote pass password prompt mget *.jpg quit END_SCRIPT exit 0 cd ../ (2 Replies)
Discussion started by: walforum
2 Replies

9. Shell Programming and Scripting

Help on FTP download using UNIX script

Hi guys, I'm new on this forum and on UNIX. Can somebody help in writing a script to download a file from an FTP server and validating if there is a file to download. If there is a file, I would send it to a mail recipient and if not I would generate an error log. Thanks in advance!:D (1 Reply)
Discussion started by: rjay_45
1 Replies

10. Shell Programming and Scripting

script for download files from ftp site

I'm new to scripting. I'm trying to write a script to download files from ftp site, the following is the script and the message i get after running the script. no files were downloaded :( Thanks advance! script: #!/usr/bin/ksh DAY=`date --date="-1 days" +%y%m%d` ftp -v -n "ftp.address" <<... (5 Replies)
Discussion started by: tiff-matt
5 Replies
Login or Register to Ask a Question