Hello All,

I am currently trying to use either WGET or CURL to automate some HTTP functions that otherwise required a lot of repeated manual steps.

From looking at the MAN pages from CURL and WGET there is some basic information concerning HTTP POST but nothing which explains exactly which steps would be needed to include a full update.

Looking at a packet trace from a POST the following content is usually sent to the webserver to get the achieved results:

POST /XRDTool/update/Update.do HTTP/1.1
Host: <snip>
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-us) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10
Content-Type: application/x-www-form-urlencoded
Referer: http://<snip>
Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Origin: <snip>
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Cookie: JSESSIONID=<snip>6; maintab=tcontent1; CP_GUTC=<snip>
Content-Length: 699
Connection: keep-alive

Number=<truncated>

You need to send a cookie to authenticate these pages it seems. You also have to get the cookie in the first place. Check out wget's --save-cookies, --load-cookies, and --keep-session-cookies settings.

From wget's manual page:

		   # Log in to the server.  This can be done only once.
		   wget --save-cookies cookies.txt \
			--post-data 'user=foo&password=bar' \
			http://server.com/auth.php

		   # Now grab the page or pages we care about.
		   wget --load-cookies cookies.txt \
			-p http://server.com/interesting/article.php

Here is an example for using curl to post data (may need the -c option to use cookies). The formfind.pl script I linked to in that post makes it much easier to find out what information has to be posted to log in with curl or wget.