Today (Saturday) We will make some minor tuning adjustments to MySQL.

You may experience 2 up to 10 seconds "glitch time" when we restart MySQL. We expect to make these adjustments around 1AM Eastern Daylight Saving Time (EDT) US.


Using cURL to submit a post form


Login or Register to Reply

 
Thread Tools Search this Thread
# 1  
Using cURL to submit a post form

I am trying to write a shell script to use curl in order to automate downloading data from a website. The URL with the post form is here: http://try-db.org/de/InfoBySpecies.php .

I have a list of about 1800 different species I want to check. For Example, choose the first species and use the Queries option "Measurement table ...". The resulting page will contain a html table with the data. Getting to this stage would be nice progress, but eventually this script needs to parse out the download link and download the resulting tab delimited file.

Command:
curl --data "species=29&OutTab=tab1&submit=Submit&dummy=" http://try-db.org/de/InfoBySpecies.php

Any suggestions would be appreciated.
Cheers.
# 2  
Take a look at the HTML of the resulting page. Getting the table is just a similar form post to DnldTraitMeas.php with an id1 value which is the species number you want.

Considering the species list alone is several megabytes in size, it makes sense to avoid downloading from that main page when it can be avoided.
# 3  
Thank you for your help. Much appreciated.

I was successfully able to submit my post to with this command: curl -d id1=29 -d submit=Download http://try-db.org/de/DnldTraitMeas.php > 29result.html

The resulting page has the filename of the temporary file that gets created. I can then script out the url to wget the actual tab delimited data.

Cheers
This User Gave Thanks to hansvg For This Post:
Login or Register to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

More UNIX and Linux Forum Topics You Might Find Helpful
CURL - Post Form Isssue ( sequel )
Fred13
Hi, I write a new thread to discuss about my closed topic with new information ( /280990-curl-post-form-issue.html ) The previous post was closed because of missing informations, I didn't have access yet to server logs. ----------------------------------------------------------------------...... Web Development
4
Web Development
Curl - post form issue
Fred13
I'm having an issue with curl post form, I dont' understand what I'm mising. I would like to send a post command login/password to a form, quite simple in the paper. URL : http: // <myebsite> / login Here the form source code : <form action="/login_check" method="post"> <input...... Web Development
3
Web Development
Curl Script - Post a file (multipart/form-data)
Anooja G
Hi All, I am trying to post an xml file (utf-16 encoded) using curl to a REST service. The REST service is expecting 'multipart/form-data' content type. curl -k -i -H "Content-Type=multipart/form-data" -F "filename=@file.xml;type=text/xml" -X POST -u <username>:<password> <endPointURL> ...... Shell Programming and Scripting
0
Shell Programming and Scripting
Can't submit a form.
tip78
hello my script is submitting POST-data to a site (its not my first script, i've done these before many times (include parsing scripts) but this one is tough) so the problem is i'm submitting a form with firefox and in firebug i see WHAT exactly i'm submitting then when i do EXACTLY the...... Shell Programming and Scripting
28
Shell Programming and Scripting
maintaining form post session using curl
Allasso
Hello, I have searched for a good part of the day, but cannot seem to find an answer to this. I hope this is an appropriate forum for my question. I am wanting to download a blog web page using a script. It requires username and password login. I have tried the following (an example) to no...... Shell Programming and Scripting
1
Shell Programming and Scripting

Featured Tech Videos