Curl command to download multiple files with a file prefix


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Curl command to download multiple files with a file prefix
# 1  
Old 05-18-2016
Curl command to download multiple files with a file prefix

I am using the below curl command to download a single file from client server and it is working as expected

Code:
curl  --ftp-ssl -k -u ${USER}:${PASSWD} ftp://${HOST}:${PORT}/path/to/${FILE}  --output ${DEST}/${FILE}

let say the client has 3 files hellofile.101, hellofile.102, hellofile.103 and I want to download all the files starting with name 'hellofile'.

Can someone please provide me the command for that.

Thanks a lot in advance.
# 2  
Old 05-19-2016
Quote:
Originally Posted by r@v!7*7@
I am using the below curl command to download a single file from client server and it is working as expected

Code:
curl  --ftp-ssl -k -u ${USER}:${PASSWD} ftp://${HOST}:${PORT}/path/to/${FILE}  --output ${DEST}/${FILE}

let say the client has 3 files hellofile.101, hellofile.102, hellofile.103 and I want to download all the files starting with name 'hellofile'.

Can someone please provide me the command for that.

Thanks a lot in advance.
Please, try:
Code:
curl  --ftp-ssl -k -u ${USER}:${PASSWD} "ftp://${HOST}:${PORT}/path/to/hellofile.10[1-3]"  --output ${DEST}/hellofile.10#1

# 3  
Old 05-19-2016
Thanks for you reply.

Actually to make my question more clear.
The client server might have n no. of such files starting with name 'hellofile'. For example hellofile.101, hellofile.102, hellofile.201, hellofile.202,
hellofile.801, hellofile.802 and so on.

So the command that you suggested will not be able to download all these file.
# 4  
Old 05-19-2016
It's kind of funny, since it's usually the other way around: I don't think curl has the capacity for this, but the underlying FTP protocol ought to:

Code:
$ ls qwert*
ls: cannot access qwert*: No such file or directory

$ ftp -p -i somesite <<EOF
user username
password password
cd asdf
mget qwert*
quit
EOF

$ ls qwert*
qwert1  qwert2  qwert3  qwert4  qwert5

$

You may need to use ftps if you have it.

If sftp is available on your server (a totally different protocol from FTP and FTPS, but these days almost more common) it should work nearly the same way here:

Code:
sftp server <<EOF
cd folder
mget prefix*
quit
EOF

...except you'll need to arrange keys to login without a password, as it will not accept forcefed passwords from a script for security reasons. see SSH login without password

Last edited by Corona688; 05-19-2016 at 02:33 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Curl parallel download file list

Hello guys, first post sorry if I did some mess here =) Using Ubuntu 14.04lts 64bits server version. I have a list (url.list) with only URLs to download, one per line, that looks like this: http://domain.com/teste.php?a=2&b=3&name=1 http://domain.com/teste.php?a=2&b=3&name=2 ...... (6 Replies)
Discussion started by: tonispa
6 Replies

2. UNIX for Dummies Questions & Answers

How to download files matching pattern from FTP using CURL or WGET?

Hi, For an order I requested, the provider has uploaded a tar file in public FTP site which internally has tons of files (compressed) and I need to download files that follows particular pattern which would be few hundreds. Note: The order can't be requested for files that follows the... (7 Replies)
Discussion started by: Amalan
7 Replies

3. Shell Programming and Scripting

Curl ftp ssl download files

Hello all, I have been struggling with this issue on and off for a couple of weeks now and I just got it all working, so I wanted to share my findings in case some other poor soul needs to know how. First some background on what I'm doing. I am uploading files to different directories based on... (0 Replies)
Discussion started by: msjkadams
0 Replies

4. Shell Programming and Scripting

How to download file without curl and wget

Hi I need a Shell script that will download a zip file every second from a http server but i can't use neither curl nor wget. Can anyone will help me go about this task ??? Thanks!! (1 Reply)
Discussion started by: rubber08
1 Replies

5. UNIX for Advanced & Expert Users

Help with using curl to download files from https

Hi I'm trying to download an xml file from a https server using curl on a Linux machine with Ubuntu 10.4.2 I am able to connect to the remote server with my username and password but the output is only "Virtual user <username> logged in". I am expecting to download the xml file. My output... (4 Replies)
Discussion started by: henryN
4 Replies

6. UNIX for Advanced & Expert Users

Curl Command to download remote certificate

Hi, I have Apache running with the certificates installed. I need a "unix curl" command to download and display remote server certificate. Kindly help. Note: Apache has ""SSLVerifyClient require" set in its configuration. (3 Replies)
Discussion started by: mohtashims
3 Replies

7. Shell Programming and Scripting

curl script to download files from Secured HTTPS server?

curl -# -v -d "sendusername=myname&password=mypassword&wheretogo=download.php" -L -o test.zip http://www.ims-dm.com/cgi/securedownload.php?p=HIREFTPM\&prodtype=hire/test.zip * About to connect() to www.ims-dm.com port 80 * Trying 209.61.193.139... connected * Connected to www.ims-dm.com... (1 Reply)
Discussion started by: laknar
1 Replies

8. Shell Programming and Scripting

FAQ how to download multiple specific files via command line

Hi, This evening i would like to download multiple pcap captures files in the wireshark wiki sites. My aim is to download the capture files .pcap .cap and etc on the wireshark site SampleCaptures - The Wireshark Wiki. i already used wget, lynx, htget but still their problem downloading..it seems... (1 Reply)
Discussion started by: jao_madn
1 Replies

9. Shell Programming and Scripting

Shell Script for Upload/download files using cURL

hi please help me out here, i want to use curl command in shell script to test web pages, what i have is an opening page, when i click on a button on opening page, the next page comes up and then i have to upload a file n then click another button to submit and then comes the output page,... (2 Replies)
Discussion started by: Olivia
2 Replies

10. UNIX for Dummies Questions & Answers

Removing prefix from multiple files and renaming file extension

Hello i have the files in this format pdb1i0t.ent pdb1lv7.ent pdb1pp6.ent pdb1tj2.ent pdb1xg2.ent pdb2b4b.ent pdb2ewe.ent Now i have to remove the prefix pdb from all the files and also i need to change the extension of .ent to .txt The new file should look like this ... (3 Replies)
Discussion started by: empyrean
3 Replies
Login or Register to Ask a Question