Sponsored Content
Top Forums Shell Programming and Scripting Same time ftp download in perl multiple sites. Post 302277463 by kofs79 on Friday 16th of January 2009 10:54:52 AM
Old 01-16-2009
I do have a file where I have all the hosts IP PWD and OS type stored. I called it sam_sites.pl.
Code:
AA|123.456.789.100|PASSWORD|Windows
BB|321.654.987.100|PASSWORD|Unix
CC|135.790.864.123|PASSWORD|Unix


I read in the file into an array @site

Code:
open(SITES_F, "sam_sites.pl") || die ("Missing sam_sites.pl file!");
@site=<SITES_F>;
close(SITES_F);
$NLETNUM = scalar(@site);

I then have a foreach loop where I take each line and set the variables to host,IP,PWD and OS. Then I start the ftp transfer.
Code:
foreach $location_n (@site)
{
chomp($location_n);
($s_name,$s_ip,$s_pwd,$s_sys)=split(/\|/,$location_n);
$thr = threads->create(\&ftp_go_and_get, $s_name, $s_ip, $s_pwd, $s_sys);
$thr->join();
}

So I guess one of my questions still is will threads do what I I wanting with d/l the files at the same time or do I need to run a perl script in background or fork()?

I appreciate your help.
 

9 More Discussions You Might Find Interesting

1. SuSE

When a FTP download is completed

I receive files by FTP in an input directory. Every 10 seconds a process checks if some new file has arrived to the input directory and then the process moves them to another directory to be read by a second process. The problem is that there is no way to know if the download has finished... (11 Replies)
Discussion started by: Javi
11 Replies

2. HP-UX

Ftp Download Problem

FROM WINDOWS , WHILE GETTING DATA FROM HP-UNIX SERVER USING FTP, I AM ABLE TO GET DATA ONLY LESS THAN 4GB FILE, PLZ HELP ME HOW TO GET MORE MORE THAN 4GB FILE. (4 Replies)
Discussion started by: niranjan
4 Replies

3. Shell Programming and Scripting

Corrupted Download with Net::FTP (perl)

Hello, I am trying to download several files from a remote FTP server using Net::FTP from within a perl script. The files download alright, except they appear to be corrupted. I know this because once after downloading, I use bzcat to pipe the files to a split command, and bzcat complains saying... (3 Replies)
Discussion started by: amcrisan
3 Replies

4. Shell Programming and Scripting

ftp multiple files at the same time

Hi All, I am using ncftpput in one of my c-shell script to upload files to a remote location. The source files keep coming continuosly and to upload each file ncftpput opens a new connection everytime. It means ncftp uploads the file1 than file2 than file3 .... There is gap 20-25 secs between... (10 Replies)
Discussion started by: sraj142
10 Replies

5. Shell Programming and Scripting

Please suggest the Sites for perl script beginners for better understanding

I am begginer to perl scripting, i like to learn all the functionality of the perl scrpting , Could you please help me on this :confused::confused: (2 Replies)
Discussion started by: jothi basu
2 Replies

6. Programming

Multiple sites running off same code

I m interested in making multiple websites, all with the same basic functionality but will have different designs or templates. is there a way they can all run off the same basic underlying code, but use different templates. what is the best method of doing this. (1 Reply)
Discussion started by: AimyThomas
1 Replies

7. Shell Programming and Scripting

Ftp multiple files one at a time

Hi, I have a problem while ftp'ing zip files one after the other from linux source directory to a ftp host machine. here is the shell script: #!/bin/ksh dir=data/dir1/dir2 # this dir is linux source directory where zip files located. rmtdir='/home/' echo $dir for i in /$dir/*; do if ;... (7 Replies)
Discussion started by: raj78
7 Replies

8. Shell Programming and Scripting

FTP download using perl script

Need assistance I have a script which i can download the files from ftp server using perl . But i want to download multiple files at a time #!/usr/bin/perl -w use Net::FTP; $ftp = Net::FTP->new("ftp.ncdc.noaa.gov"); $ftp->login('username', 'password'); $ftp->cwd("<dir>");... (9 Replies)
Discussion started by: ajayram_arya
9 Replies

9. Shell Programming and Scripting

Read latest file name with a date and time and then download from FTP

Hi All, Please help. I have requirement to read the file / folder based on the latest date and download the file and folder. There will be files and folders in the location like 20140630-144422 20140630-144422.csv 20140707-182653 20140707-182653.csv 20140710-183153... (7 Replies)
Discussion started by: Praveen Pandit
7 Replies
AXEL(1) 						      General Commands Manual							   AXEL(1)

NAME
Axel - A light download accelerator for Linux. SYNOPSIS
axel [OPTIONS] url1 [url2] [url...] DESCRIPTION
Axel is a program that downloads a file from a FTP or HTTP server through multiple connection, each connection downloads its own part of the file. Unlike most other programs, Axel downloads all the data directly to the destination file, using one single thread. It just saves some time at the end because the program doesn't have to concatenate all the downloaded parts. OPTIONS
One argument is required, the URL to the file you want to download. When downloading from FTP, the filename may contain wildcards and the program will try to resolve the full filename. Multiple URL's can be specified as well and the program will use all those URL's for the download. Please note that the program does not check whether the files are equal. Other options: --max-speed=x, -s x You can specify a speed (bytes per second) here and Axel will try to keep the average speed around this speed. Useful if you don't want the program to suck up all of your bandwidth. --num-connections=x, -n x You can specify an alternative number of connections here. --output=x, -o x Downloaded data will be put in a local file with the same name, unless you specify a different name using this option. You can spec- ify a directory as well, the program will append the filename. --search[=x], -S[x] Axel can do a search for mirrors using the filesearching.com search engine. This search will be done if you use this option. You can specify how many different mirrors should be used for the download as well. The search for mirrors can be time-consuming because the program tests every server's speed, and it checks whether the file's still available. --no-proxy, -N Don't use any proxy server to download the file. Not possible when a transparent proxy is active somewhere, of course. --verbose If you want to see more status messages, you can use this option. Use it more than once if you want to see more. --quiet, -q No output to stdout. --alternate, -a This will show an alternate progress indicator. A bar displays the progress and status of the different threads, along with current speed and an estimate for the remaining download time. --header=x, -H x Add an additional HTTP header. This option should be in the form "Header: Value". See RFC 2616 section 4.2 and 14 for details on the format and standardized headers. --user-agent=x, -U x Set the HTTP user agent to use. Some websites serve different content based upon this parameter. The default value will include "Axel", its version and the platform. --help, -h A brief summary of all the options. --version, -V Get version information. NOTE
Long (double dash) options are supported only if your platform knows about the getopt_long call. If it does not (like *BSD), only the short options can be used. RETURN VALUE
The program returns 0 when the download was succesful, 1 if something really went wrong and 2 if the download was interrupted. If something else comes back, it must be a bug.. EXAMPLES
axel ftp://ftp.{be,nl,uk,de}.kernel.org/pub/linux/kernel/v2.4/linux-2.4.17.tar.bz2 This will use the Belgian, Dutch, English and German kernel.org mirrors to download a Linux 2.4.17 kernel image. axel -S4 ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-2.4.17.tar.bz2 This will do a search for the linux-2.4.17.tar.bz2 file on filesearching.com and it'll use the four (if possible) fastest mirrors for the download. (Possibly including ftp.kernel.org) (Of course, the commands are a single line, but they're too long to fit on one line in this page.) FILES
/etc/axelrc System-wide configuration file. Note that development versions place this file in /usr/local/etc. ~/.axelrc Personal configuration file These files are not documented in a man-page, but the example file which comes with the program contains enough information, I hope. The position of the system-wide configuration file might be different. COPYRIGHT
Axel is Copyright 2001-2002 Wilmer van der Gaast. BUGS
Please report bugs at https://alioth.debian.org/tracker/?group_id=100070&atid=413085. AUTHORS
Wilmer van der Gaast. <wilmer@gaast.net> AXEL(1)
All times are GMT -4. The time now is 05:50 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy