Using cURL to submit a post form


Login or Register to Reply

 
Thread Tools Search this Thread
# 1  
Old 07-24-2012
Using cURL to submit a post form

I am trying to write a shell script to use curl in order to automate downloading data from a website. The URL with the post form is here: http://try-db.org/de/InfoBySpecies.php .

I have a list of about 1800 different species I want to check. For Example, choose the first species and use the Queries option "Measurement table ...". The resulting page will contain a html table with the data. Getting to this stage would be nice progress, but eventually this script needs to parse out the download link and download the resulting tab delimited file.

Command:
curl --data "species=29&OutTab=tab1&submit=Submit&dummy=" http://try-db.org/de/InfoBySpecies.php

Any suggestions would be appreciated.
Cheers.
# 2  
Old 07-24-2012
Take a look at the HTML of the resulting page. Getting the table is just a similar form post to DnldTraitMeas.php with an id1 value which is the species number you want.

Considering the species list alone is several megabytes in size, it makes sense to avoid downloading from that main page when it can be avoided.
# 3  
Old 07-24-2012
Thank you for your help. Much appreciated.

I was successfully able to submit my post to with this command: curl -d id1=29 -d submit=Download http://try-db.org/de/DnldTraitMeas.php > 29result.html

The resulting page has the filename of the temporary file that gets created. I can then script out the url to wget the actual tab delimited data.

Cheers
This User Gave Thanks to hansvg For This Post:
Corona688 (07-25-2012)
Login or Register to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

More UNIX and Linux Forum Topics You Might Find Helpful
CURL - Post Form Isssue ( sequel ) Fred13 Web Development 4 2 Weeks Ago 12:33 PM
Curl - post form issue Fred13 Web Development 3 2 Weeks Ago 06:11 AM
Trying to submit web form content to a shell script Paul Walker Shell Programming and Scripting 0 05-31-2018 12:37 PM
Exec submit form in bash script Herbert Shell Programming and Scripting 0 12-15-2017 06:57 AM
Html form to submit data to bash script robertkwild Programming 3 07-20-2017 08:31 AM
How to use cURL to download web page with autentification (form)? MarcelOrMittal UNIX for Beginners Questions & Answers 1 08-04-2016 12:15 PM
Curl to hit the submit button Kochappa Shell Programming and Scripting 1 09-01-2014 09:54 AM
Curl Script - Post a file (multipart/form-data) Anooja G Shell Programming and Scripting 0 04-28-2014 09:08 AM
How to submit form on an php webpage from command line? postcd UNIX for Dummies Questions & Answers 5 02-07-2014 12:19 PM
Can't submit a form. tip78 Shell Programming and Scripting 28 02-13-2013 02:08 PM
Submit using curl limadario Ubuntu 8 06-06-2010 08:04 PM
Posting data to a form using curl DeCoTwc UNIX for Dummies Questions & Answers 0 05-30-2010 04:34 AM
using wget to submit a form on linksys router mike909 Shell Programming and Scripting 1 04-11-2009 04:32 PM
PHP Help with Form Submit nck Shell Programming and Scripting 3 03-26-2009 12:14 PM
maintaining form post session using curl Allasso Shell Programming and Scripting 1 01-13-2009 06:26 PM