Posting Multiple Files using cURL


 
Thread Tools Search this Thread
Operating Systems Linux Red Hat Posting Multiple Files using cURL
# 1  
Old 03-26-2012
Posting Multiple Files using cURL

Hi,

I am currently in the process of creating a script that will use cURL to HTTP POST the contents of a file on the server.

I am able to run the command;

Code:
curl -d @/path/to/file/filename.dat "https://posting.server.com/test.aspx"

to post a single file and this works as expected.

The trouble I am experiencing is that I will have a folder that all of the output files which need to be posted will be going into, and the name of each file will change sequentially (number range e.g. 4000001.dat 4000002.dat).

I have created a script which the command;

Code:
curl -d @/path/to/file/*.dat "https://posting.server.com/test.aspx"

But this returns the error;

Code:
Warning: Couldn't read data from file 
Warning: "/path/to/file/*.dat", this makes an 
Warning: empty POST.

I had a look around and cannot see how to perhaps get a list of each file name within the directory, then run my original command for each file name.

e.g.

Files;
4000001.dat
4000002.dat

script should then run commands;

curl -d @/path/to/file/4000001.dat "https://posting.server.com/test.aspx"
curl -d @/path/to/file/4000002.dat "https://posting.server.com/test.aspx"

This is the way i'd imagine it working, but perhaps somebody knows an equivalent to using '*' within the cURL command.

I think I've explained everything, but then again it is a Monday Smilie

Last edited by jim mcnamara; 03-26-2012 at 01:47 PM..
# 2  
Old 03-26-2012
The @ prevents it from working because the shell takes it literally, and finds no files @/... Also, curl expects the list to be comma-separated.

You could make the list first, then put it in:

Code:
# Save the list of files in $1, $2, ...
# This will overwrite the current values of $1, $2, ...
set -- /path/to/file/*.dat
# Split on , so the list you generate with "$*" will be comma-separated
OLDIFS="$IFS"
IFS=","

curl -d @"$*" "https://posting.server.com/test.aspx"
IFS="$OLDIFS"


Last edited by Corona688; 03-26-2012 at 02:05 PM..
# 3  
Old 03-28-2012
Thanks for your reply.

I have taken a look and had a go at this, but unfortunately the files which I need to send must keep their original file name as they are order numbers.

Is it possible to create a script that will run my command for each file in the folder, changing the file name in the command each time?
# 4  
Old 03-28-2012
Quote:
Originally Posted by MarkPaxo
Thanks for your reply.

I have taken a look and had a go at this, but unfortunately the files which I need to send must keep their original file name as they are order numbers.
They do keep their original file names. Take another look at what I did. I'm not replacing files.

Code:
$ touch 01 02 03
$ echo 0*
01 02 03

$ set -- 0*
$ echo "$*"
01 02 03

$ IFS=","
$ echo "$*"
01,02,03

$

I'm just abusing set -- and IFS to produce a list delimited with commas, exactly the way curl wants it.
# 5  
Old 04-05-2012
Hi,

Thanks. I will try it again today. Have not had a chance to look at this properly for a while.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Check status of long running multiple curl commands in shell script

I am working on script. it reads a file which contains multiple lines Ex; curl --write-out %{http_code} --silent --output /dev/null http://hostname:port/input=1 curl --write-out %{http_code} --silent --output /dev/null http://hostname:port/input=2 curl --write-out %{http_code} --silent ... (2 Replies)
Discussion started by: oraclermanpt
2 Replies

2. Shell Programming and Scripting

Curl - upload multiple attachment arrays to server webpage

The html page of the form data is as below <form name="uploadform" id="uploadform" action="htmlupload.php" enctype="multipart/form-data" method="post"> <table class="tborder" cellpadding="6" cellspacing="1" border="0" width="100%" align="center"> <tr> <td class="tcat"> Upload Files ... (0 Replies)
Discussion started by: jaango123
0 Replies

3. Shell Programming and Scripting

Loop to curl multiple ips to ipinfo.io

I am rather new to shell scripting and currently taking a linux course. Im having some troubles writing a loop to curl multiple ips in mutiple access logs to the site ipinfo.io and push the output to a text file for easy viewing and removing duplicates. So far i have these simple lines cat... (1 Reply)
Discussion started by: kjcraig77
1 Replies

4. Shell Programming and Scripting

Curl command to download multiple files with a file prefix

I am using the below curl command to download a single file from client server and it is working as expected curl --ftp-ssl -k -u ${USER}:${PASSWD} ftp://${HOST}:${PORT}/path/to/${FILE} --output ${DEST}/${FILE} let say the client has 3 files hellofile.101, hellofile.102, hellofile.103 and I... (3 Replies)
Discussion started by: r@v!7*7@
3 Replies

5. UNIX for Dummies Questions & Answers

How to download files matching pattern from FTP using CURL or WGET?

Hi, For an order I requested, the provider has uploaded a tar file in public FTP site which internally has tons of files (compressed) and I need to download files that follows particular pattern which would be few hundreds. Note: The order can't be requested for files that follows the... (7 Replies)
Discussion started by: Amalan
7 Replies

6. Shell Programming and Scripting

Curl ftp ssl download files

Hello all, I have been struggling with this issue on and off for a couple of weeks now and I just got it all working, so I wanted to share my findings in case some other poor soul needs to know how. First some background on what I'm doing. I am uploading files to different directories based on... (0 Replies)
Discussion started by: msjkadams
0 Replies

7. UNIX for Advanced & Expert Users

Help with using curl to download files from https

Hi I'm trying to download an xml file from a https server using curl on a Linux machine with Ubuntu 10.4.2 I am able to connect to the remote server with my username and password but the output is only "Virtual user <username> logged in". I am expecting to download the xml file. My output... (4 Replies)
Discussion started by: henryN
4 Replies

8. Shell Programming and Scripting

Shell Script for Upload/download files using cURL

hi please help me out here, i want to use curl command in shell script to test web pages, what i have is an opening page, when i click on a button on opening page, the next page comes up and then i have to upload a file n then click another button to submit and then comes the output page,... (2 Replies)
Discussion started by: Olivia
2 Replies

9. UNIX for Dummies Questions & Answers

Using AWK: Extract data from multiple files and output to multiple new files

Hi, I'd like to process multiple files. For example: file1.txt file2.txt file3.txt Each file contains several lines of data. I want to extract a piece of data and output it to a new file. file1.txt ----> newfile1.txt file2.txt ----> newfile2.txt file3.txt ----> newfile3.txt Here is... (3 Replies)
Discussion started by: Liverpaul09
3 Replies

10. UNIX for Dummies Questions & Answers

Posting data to a form using curl

Hello all. I have an incredible number of servers that I need to change a parameter on using a web interface. I'd like to be able to do this via curl, but I'm having some trouble. I filled out the form and hit update while snooping (tcpdump) my interface. That gave the the following as what is... (0 Replies)
Discussion started by: DeCoTwc
0 Replies
Login or Register to Ask a Question