01-16-2009
Same time ftp download in perl multiple sites.
I currently have a perl script that have to retreive a single file from 20+ sites every 10 min. Right now it will ftp to site1 and download and continue up until site20. I am trying to get this to run all the ftp d/l at the same time. This is where I have my problem, I can't get it to work.
Sample of sam_sites.pl file :
AA|123.456.789.100|PASSWORD|Windows
BB|321.654.987.100|PASSWORD|Unix
CC|135.790.864.123|PASSWORD|Unix
Code from ret_sam.pl file :
#!/usr/bin/perl
use Net::FTP;
use Switch;
use threads;
use threads::shared;
$version = "0.00.0001";
$RETSAMC = `ps -ef | grep ret_sam | grep -v grep | wc -l`;
$RETSAMS = "/home/fox/SAM/RET_SAM_RUN";
$HOMEDIR = "/home/fox/SAM/sites/test";
open(SITES_F, "sam_sites.pl") || die ("Missing sam_sites.pl file!");
@site=<SITES_F>;
close(SITES_F);
$NLETNUM = scalar(@site);
foreach $location_n (@site)
{
chomp($location_n);
($s_name,$s_ip,$s_pwd,$s_sys)=split(/\|/,$location_n);
$thr = threads->create(\&ftp_go_and_get, $s_name, $s_ip, $s_pwd, $s_sys);
$thr->join();
}
sub ftp_go_and_get {
if (lc($_[3]) eq "windows"){
$f_user = "fox\@openfox";
$directory = "SAM";
} elsif (lc($_[3]) eq "unix"){
$f_user = "fox";
$directory = "/home/fox/SAM";
}
$ftp=Net::FTP->new($_[1],Timeout=>60) or $newerr=1;
push @ERRORS, "Can't ftp to $_[1]($_[0]): $!\n" if $newerr;
myerr() if $newerr;
print "Connected to $_[1]($_[0])\n";
$ftp->login("$f_user","$_[2]") or $newerr=1;
print "Trying to get in and grab file.\n";
push @ERRORS, "Can't login to $_[1]($_[0]): $!\n" if $newerr;
# $ftp->quit if $newerr;
myerr() if $newerr;
print "Logged in\n";
$ftp->cwd("$directory") or $newerr=1;
push @ERRORS, "Can't find a spot to take dump.\n" if $newerr;
myerr() if $newerr;
# $ftp->quit if $newerr;
#$f_size=$ftp->size("@_[0]") or $newerr=1;
#print "The size of this file is $f_size\n";
# push @ERRORS, "Can't get size of file.\n" if $newerr;
# myerr() if $newerr;
# $ftp->quit if $newerr;
@files=$ftp->get($_[0], "$HOMEDIR/$_[0]") or $newerr=1;
push @ERRORS, "Can't get file $_[0]\n" if $newerr;
myerr() if $newerr;
print "Got file for $_[0]\n";
foreach(@files) {
print "I took a dump in $_\n";
}
$ftp->quit;
}
sub myerr {
print "Error: \n";
print @ERRORS;
exit 0;
}
I am looking for any suggestions or constructive critism that can help me out here. Thanks in advance.
9 More Discussions You Might Find Interesting
1. SuSE
I receive files by FTP in an input directory. Every 10 seconds a process checks if some new file has arrived to the input directory and then the process moves them to another directory to be read by a second process.
The problem is that there is no way to know if the download has finished... (11 Replies)
Discussion started by: Javi
11 Replies
2. HP-UX
FROM WINDOWS , WHILE GETTING DATA FROM HP-UNIX SERVER USING FTP, I AM ABLE TO GET DATA ONLY LESS THAN 4GB FILE, PLZ HELP ME HOW TO GET MORE MORE THAN 4GB FILE. (4 Replies)
Discussion started by: niranjan
4 Replies
3. Shell Programming and Scripting
Hello,
I am trying to download several files from a remote FTP server using Net::FTP from within a perl script. The files download alright, except they appear to be corrupted. I know this because once after downloading, I use bzcat to pipe the files to a split command, and bzcat complains saying... (3 Replies)
Discussion started by: amcrisan
3 Replies
4. Shell Programming and Scripting
Hi All,
I am using ncftpput in one of my c-shell script to upload files to a remote location. The source files keep coming continuosly and to upload each file ncftpput opens a new connection everytime. It means ncftp uploads the file1 than file2 than file3 .... There is gap 20-25 secs between... (10 Replies)
Discussion started by: sraj142
10 Replies
5. Shell Programming and Scripting
I am begginer to perl scripting, i like to learn all the functionality of the perl scrpting , Could you please help me on this :confused::confused: (2 Replies)
Discussion started by: jothi basu
2 Replies
6. Programming
I m interested in making multiple websites, all with the same basic functionality but will have different designs or templates. is there a way they can all run off the same basic underlying code, but use different templates. what is the best method of doing this. (1 Reply)
Discussion started by: AimyThomas
1 Replies
7. Shell Programming and Scripting
Hi,
I have a problem while ftp'ing zip files one after the other from linux source directory to a ftp host machine. here is the shell script:
#!/bin/ksh
dir=data/dir1/dir2 # this dir is linux source directory where zip files located.
rmtdir='/home/'
echo $dir
for i in /$dir/*; do
if ;... (7 Replies)
Discussion started by: raj78
7 Replies
8. Shell Programming and Scripting
Need assistance
I have a script which i can download the files from ftp server using perl . But i want to download multiple files at a time
#!/usr/bin/perl -w
use Net::FTP;
$ftp = Net::FTP->new("ftp.ncdc.noaa.gov");
$ftp->login('username', 'password');
$ftp->cwd("<dir>");... (9 Replies)
Discussion started by: ajayram_arya
9 Replies
9. Shell Programming and Scripting
Hi All,
Please help.
I have requirement to read the file / folder based on the latest date and download the file and folder.
There will be files and folders in the location like
20140630-144422
20140630-144422.csv
20140707-182653
20140707-182653.csv
20140710-183153... (7 Replies)
Discussion started by: Praveen Pandit
7 Replies
LEARN ABOUT DEBIAN
bup-ftp
bup-ftp(1) General Commands Manual bup-ftp(1)
NAME
bup-ftp - ftp-like client for navigating bup repositories
SYNOPSIS
bup ftp
DESCRIPTION
bup ftp is a command-line tool for navigating bup repositories. It has commands similar to the Unix ftp(1) command. The file hierarchy is
the same as that shown by bup-fuse(1) and bup-ls(1).
Note: if your system has the python-readline library installed, you can use the <tab> key to complete filenames while navigating your
backup data. This will save you a lot of typing.
COMMANDS
The following commands are available inside bup ftp:
ls [-s] [-a] [path]
print the contents of a directory. If no path argument is given, the current directory's contents are listed. If -a is given, also
include hidden files (files which start with a . character). If -s is given, each file is displayed with its hash from the bup ar-
chive to its left.
cd dirname
change to a different working directory
pwd print the path of the current working directory
cat filenames...
print the contents of one or more files to stdout
get filename localname
download the contents of filename and save it to disk as localname. If localname is omitted, uses filename as the local name.
mget filenames...
download the contents of the given filenames and stores them to disk under the same names. The filenames may contain Unix filename
globs (*, ?, etc.)
help print a list of available commands
quit exit the bup ftp client
EXAMPLE
$ bup ftp
bup> ls
mybackup/ yourbackup/
bup> cd mybackup/
bup> ls
2010-02-05-185507@ 2010-02-05-185508@ latest@
bup> cd latest/
bup> ls
(...etc...)
bup> get myfile
Saving 'myfile'
bup> quit
SEE ALSO
bup-fuse(1), bup-ls(1), bup-save(1), bup-restore(1)
BUP
Part of the bup(1) suite.
AUTHORS
Avery Pennarun <apenwarr@gmail.com>.
Bup unknown- bup-ftp(1)