09-03-2010
I would suggest you see about using
lftp, possibly within your shellscript.
It has an interesting feature named "mirror", specifically where you can recurse directories and download files matching a glob pattern, such as "*.fna"
LFTP - sophisticated file transfer program
HTH
dan
10 More Discussions You Might Find Interesting
1. SuSE
I receive files by FTP in an input directory. Every 10 seconds a process checks if some new file has arrived to the input directory and then the process moves them to another directory to be read by a second process.
The problem is that there is no way to know if the download has finished... (11 Replies)
Discussion started by: Javi
11 Replies
2. HP-UX
FROM WINDOWS , WHILE GETTING DATA FROM HP-UNIX SERVER USING FTP, I AM ABLE TO GET DATA ONLY LESS THAN 4GB FILE, PLZ HELP ME HOW TO GET MORE MORE THAN 4GB FILE. (4 Replies)
Discussion started by: niranjan
4 Replies
3. HP-UX
from windows ftp tool even you are not root user you can delete root files (8 Replies)
Discussion started by: mkhsy
8 Replies
4. UNIX for Dummies Questions & Answers
I am trying to download from ftp server (rsync and postfix). But every time I connect to the ftp server, it prompts for USER and PASS but I don't know which username and and password to use. And I am using command based UNIX OS. (2 Replies)
Discussion started by: kumarrana
2 Replies
5. Shell Programming and Scripting
Hi all,
I'm having problems downloading files from ftp server. I have to download only those files that name starts with YYYYMMDD.But file comes like the format "20080624221035.TXT".Also how i can get list of all file names with in specified folder.
Here i paste my code
ftp -vn... (1 Reply)
Discussion started by: vaskarbasak
1 Replies
6. UNIX for Dummies Questions & Answers
Hello,
I know this is probably a very silly question for most but how to do I force curl to do active FTP downloads?
Thank you
Dallas (2 Replies)
Discussion started by: Dallasbr
2 Replies
7. Shell Programming and Scripting
Hi guys,
I'm new on this forum and on UNIX. Can somebody help in writing a script to download a file from an FTP server and validating if there is a file to download. If there is a file, I would send it to a mail recipient and if not I would generate an error log.
Thanks in advance!:D (1 Reply)
Discussion started by: rjay_45
1 Replies
8. Shell Programming and Scripting
Hi !
I am just wondering shell script to download data from ftp...
I have text file containing ftp address...looks like this
ftp://site...../filename.xyz
ftp://site...../filename.xyz
ftp://site...../filename.xyz
ftp://site...../filename.xyz
script has to read ftp address and... (8 Replies)
Discussion started by: nex_asp
8 Replies
9. Shell Programming and Scripting
Hi,
I have few files at FTP location and have written a script to download the same from ftp based on the sysdate - 1, however with the below code I am unable to download the files from FTP.
x=`TZ=CST+24 date +%Y%m%d`
mget Daily_BIH_$x_NEW.tar.gz
can anyone please help me in... (10 Replies)
Discussion started by: rramkrishnas
10 Replies
10. Shell Programming and Scripting
Hi,
I am trying to download files from FTP using below FTP code.
x=$(TZ=CST date +%Y%m%d)
host='xxx.xx.xxx.xxx'
user='userX'
pass='Password'
echo "Connecting to FTP Host -- $host....."
echo $x
a=$(ftp -v -n -i << !
open $host
user "$user" $pass
cd /rose/yellow/
mget... (6 Replies)
Discussion started by: rramkrishnas
6 Replies
LEARN ABOUT DEBIAN
web2disk
WEB2DISK(1) calibre WEB2DISK(1)
NAME
web2disk - part of calibre
SYNOPSIS
web2disk URL
DESCRIPTION
Where URL is for example http://google.com
Whenever you pass arguments to web2disk that have spaces in them, enclose the arguments in quotation marks.
OPTIONS
--version
show program's version number and exit
-h, --help
show this help message and exit
-d, --base-dir
Base directory into which URL is saved. Default is .
-t, --timeout
Timeout in seconds to wait for a response from the server. Default: 10.0 s
-r, --max-recursions
Maximum number of levels to recurse i.e. depth of links to follow. Default 1
-n, --max-files
The maximum number of files to download. This only applies to files from <a href> tags. Default is 2147483647
--delay
Minimum interval in seconds between consecutive fetches. Default is 0 s
--encoding
The character encoding for the websites you are trying to download. The default is to try and guess the encoding.
--match-regexp
Only links that match this regular expression will be followed. This option can be specified multiple times, in which case as long
as a link matches any one regexp, it will be followed. By default all links are followed.
--filter-regexp
Any link that matches this regular expression will be ignored. This option can be specified multiple times, in which case as long as
any regexp matches a link, it will be ignored. By default, no links are ignored. If both filter regexp and match regexp are speci-
fied, then filter regexp is applied first.
--dont-download-stylesheets
Do not download CSS stylesheets.
--verbose
Show detailed output information. Useful for debugging
SEE ALSO
The User Manual is available at http://manual.calibre-ebook.com
Created by Kovid Goyal <kovid@kovidgoyal.net>
web2disk (calibre 0.8.51) January 2013 WEB2DISK(1)