Sponsored Content
Top Forums Shell Programming and Scripting Using cURL to submit a post form Post 302676423 by hansvg on Tuesday 24th of July 2012 02:15:26 PM
Old 07-24-2012
Using cURL to submit a post form

I am trying to write a shell script to use curl in order to automate downloading data from a website. The URL with the post form is here: http://try-db.org/de/InfoBySpecies.php .

I have a list of about 1800 different species I want to check. For Example, choose the first species and use the Queries option "Measurement table ...". The resulting page will contain a html table with the data. Getting to this stage would be nice progress, but eventually this script needs to parse out the download link and download the resulting tab delimited file.

Command:
curl --data "species=29&OutTab=tab1&submit=Submit&dummy=" http://try-db.org/de/InfoBySpecies.php

Any suggestions would be appreciated.
Cheers.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

maintaining form post session using curl

Hello, I have searched for a good part of the day, but cannot seem to find an answer to this. I hope this is an appropriate forum for my question. I am wanting to download a blog web page using a script. It requires username and password login. I have tried the following (an example) to no... (1 Reply)
Discussion started by: Allasso
1 Replies

2. Shell Programming and Scripting

PHP Help with Form Submit

Hi, I have a custom HTML form that has a couple radio buttons and a text field that requires a number. I'm not a php programmer and could use some help with putting together php code to calculate a total based on the radio button selection and the text field number. ... (3 Replies)
Discussion started by: nck
3 Replies

3. Shell Programming and Scripting

using wget to submit a form on linksys router

I'm trying to use wget to submit a form. I have tried to dig out what is actually being "posted" and where, using tamperdata (see below). http://ubuntuforums.org/attachment.php?attachmentid=109123&d=1239224127 Here is my wget command: wget --http-user=xyz --http-password=xyz... (1 Reply)
Discussion started by: mike909
1 Replies

4. Ubuntu

Submit using curl

I'm trying to upload a file to a page using curl and after uploading that file i want to store the redirecting page so i can download the results. I'm using the command: curl "http://apps.gdgps.net/kag_upload.php?kag_type=static&kag_frequency=dual&kag_latency=accurate&... (8 Replies)
Discussion started by: limadario
8 Replies

5. Shell Programming and Scripting

Can't submit a form.

hello my script is submitting POST-data to a site (its not my first script, i've done these before many times (include parsing scripts) but this one is tough) so the problem is i'm submitting a form with firefox and in firebug i see WHAT exactly i'm submitting then when i do EXACTLY the... (28 Replies)
Discussion started by: tip78
28 Replies

6. Shell Programming and Scripting

Curl Script - Post a file (multipart/form-data)

Hi All, I am trying to post an xml file (utf-16 encoded) using curl to a REST service. The REST service is expecting 'multipart/form-data' content type. curl -k -i -H "Content-Type=multipart/form-data" -F "filename=@file.xml;type=text/xml" -X POST -u <username>:<password> <endPointURL> ... (0 Replies)
Discussion started by: Anooja G
0 Replies

7. Shell Programming and Scripting

Curl to hit the submit button

Hello, I am looking to hit a URL using curl and click on submit button so that I can get the results. The below is the code <input name="tos_accepted" id="tos_accepted" class="button" type="submit" value="Yes, I Agree"/> <input name="tos_discarded" id="tos_discarded"... (1 Reply)
Discussion started by: Kochappa
1 Replies

8. Shell Programming and Scripting

Exec submit form in bash script

Hi All, I'm new in forum. Many congratulations to everyone for all work. I'm not an expert in bash script I've a problem with a sh file. The sh file run every t minuts and it read data from txt file and then compile form. Finally, the user, from the web browser click on send. The script... (0 Replies)
Discussion started by: Herbert
0 Replies

9. Web Development

Curl - post form issue

I'm having an issue with curl post form, I dont' understand what I'm mising. I would like to send a post command login/password to a form, quite simple in the paper. URL : http: // <myebsite> / login Here the form source code : <form action="/login_check" method="post"> <input... (3 Replies)
Discussion started by: Fred13
3 Replies

10. Web Development

CURL - Post Form Isssue ( sequel )

Hi, I write a new thread to discuss about my closed topic with new information ( /280990-curl-post-form-issue.html ) The previous post was closed because of missing informations, I didn't have access yet to server logs. ----------------------------------------------------------------------... (4 Replies)
Discussion started by: Fred13
4 Replies
HXCOPY(1)							  HTML-XML-utils							 HXCOPY(1)

NAME
hxcopy - copy an HTML file and update its relative links SYNOPSIS
hxcopy [ -i old-URL ] [ -o new-URL ] [ file-or-URL [ file-or-URL ] ] DESCRIPTION
The hxcopy command copies its first argument to its second argument, while updating relative links. The input is assumed to be HTML or XHTML and may be slightly reformatted in the process. If the second argument is omitted, hxcopy writes to standard output. In this case the option -o is required. If the first argument is also omitted, hxcopy reads from standard input. In this case the option -i is required. OPTIONS
The following options are supported: -i old-URL For the purposes of updating relative links, act as if old-URL is the location from which the input is copied. If this option is omitted, the actual location of the first argument is used for calculating relative links. -o new-URL For the purposed of updating relative links, act as if new-URL is the location to which the input is copied. If this option is omitted, the actual location of the second argument is used for calculating relative links. ENVIRONMENT
To use a proxy to retrieve remote files, set the environment variables http_proxy and ftp_proxy. E.g., http_proxy="http://localhost:8080/" BUGS
Unlike the last argument of cp(1), the last argument of hxcopy must be a file, not a directory. The second argument must be a local file. Writing to a URL is not yet implemented. To work around this, replace hxcopy file.html http://example.org/file.html by hxcopy -o http://example.org/file.html file.html tmp.html and then upload tmp.html to the given URL with some other command, such as curl(1). The first argument, however, may be a URL. hxcopy will download the given file. (Currently only HTTP is supported.) EXAMPLE
Assume the HTML file foo.html contains a relative link to "../bar.html". Here are some examples of commands: hxcopy foo.html bar/foo.html The file foo.html is copied to ../bar/foo.html and the relative link to "../bar.html" becomes "../../bar.html". hxcopy foo.html ../foo.html The file foo.html is copied to ../foo.html and the relative link to "../bar.html" is rewritten as "bar.html". hxcopy -i http://my.org/dir1/foo.html -o http://my.org/foo.html file1.html file2.html The file file1.html is copied to file2.html and the relative link to "../bar.html" is rewritten as "bar.html". A command like this may be useful to update files that are later uploaded to a server. SEE ALSO
cp(1), curl(1), hxwls(1) 6.x 9 Dec 2008 HXCOPY(1)
All times are GMT -4. The time now is 11:33 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy