HTTP Query Request & Parsing returned XML output


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting HTTP Query Request & Parsing returned XML output
# 1  
Old 05-06-2005
HTTP Query Request & Parsing returned XML output

I have a PERL script from which I need to make a HTTP request to Web Servlet (Essentially a URL with variables and values like &Variable1=AAAAAA&Variable2=BBBBBBBBB&Variable3=CCCCCCC). The Web servlet returns an XML result which needs to be parsed for the contents of the result within the program.

Here is what I need to know
1) How to make this http request from PERL (What modules would I require for this request) and
2) How do I read the results from the XML results that the query returns?

Any help and or suggestions provided is greatly appreciated. Thank you.
# 2  
Old 05-06-2005
(1) You can always construct the query string from the parameters manually. It's just a matter of 'key=value' pairs with '&' in between (as you have mentioned). If you have the list of key-value pairs ready, this should be easy but you need to watch out that non-alphanumeric characters should be escaped by the %XY notation, where XY is the hex. representation of the character's ASCII value.

Or, you can use the modules URI and URI::QueryParam which I think is safer.

Code:
use URI;
use URI::QueryParam;
$u = new URI("http://someserver.com/control/servlet");
%aryParams = (
    'test1' => 'value1',
    'test2' => '$abc#def',
);
foreach (keys %aryParams) {
	$u->query_param($_, $aryParams{$_});
}
print $u->as_string() . "\n";

Output:
Code:
http://someserver.com/control/servlet?test1=value1&test2=%24abc%23def

Note that the module will do the necessary escaping for you, automatically.

Finally, use LWP::Simple to fire the HTTP request.

(2) For reading XML files, I have previously tried using XML::Simple. It builds a data structure corresponding to the XML file. Suitable for small XML files that are not so complicated but it is quite convenient.

A more complex alternative is XML::Parser, with a more-or-less SAX-like API. I haven't used this before in practice. And there are many other choices of XML parsing modules available on CPAN. Go to search.cpan.org and have a look.

I have recently worked on a PHP project with lots of reading and processing from XML files. I decided to use XPath extensively and found that I required far less code with XPath than conventional "tree-walk" mechanisms. You can extract an arbitrary node anywhere in the document with some precise conditions specified just as SQL in database queries, which is handy for more complex analysis. In Perl, an equivalent module is XML::XPath.

You need to download most of these modules as they are not in the core distribution.
# 3  
Old 05-10-2005
Thank you for the guidance and update cbkihong. We are using AIX 5.2.0 build and the sys admin says that he has installed all optional modules for PERL.
In your post you indicate three packages. URI, LWP and XML. I think I have URI::QueryParam;
LWP::Simple;
XML::Simple;

As indicated I do not have XML::Parser and XML::XPath. Can you please suggest a link on cpan where I might find these or do I need to go to IBM to obtain the above. Also can any suggested reading material for XML::XPath and XML::Parser either in book form or web-sites? Thanks again.

Jerardfjay

Last edited by jerardfjay; 05-10-2005 at 08:23 AM..
# 4  
Old 05-10-2005
A search of http://search.cpan.org/ will give you what you need - I've just checked and they're there... Smilie

Cheers
ZB
# 5  
Old 05-10-2005
Honestly speaking, to my dismay I have found some XML modules for Perl are not that mature, still (the ones I mentioned are okay). Some only implement a part of the W3C standards (though it may be actually quite usable by most people). For many modules, the documentation are not well-written and some may even mislead you into antiquated material. So I tend to go Google for bits and pieces. But I think if you have an idea of the specs, you will find that the API for these XML technologies for most programming languages are highly familiar, so resources written for other languages may also be useful. Just try the examples you can find, bits and pieces, from the Web and you will soon find it easy to follow.

I think you will find this sample chapter excerpt useful:
http://www.webreference.com/programm...perlxml/chap3/

While I did my PHP project at work I read directly the W3C spec for XPath. But that is not any pleasurable reading after all. I also read the relevant chapters of the J2EE Tutorial for info (it's for Java, of course). For a simple introduction, you may go to w3schools.com.

For modules installation, modules on CPAN can be downloaded and installed with a simple (single!) command on the command line! A past poster here has written a short tutorial for that:

https://www.unix.com/tips-and-tutorials/14246-perl-cpan-intro-newbies.html

Refer him/her here to take a look.
# 6  
Old 05-10-2005
Thank you all for the wonderful insights into the world of PERL. I am still learning new stuff everyday. This makes this site truly wonderful learning experience. Thanks again.

Jerardfjay
# 7  
Old 05-11-2005
Quote:
Originally Posted by jerardfjay
I am still learning new stuff everyday.
So are we. All. In

http://www.perl.com/pub/a/2000/06/27/perlbook.html

an active Perl community member wrote that

Quote:
You can't learn Perl in 24 hours, 21 days, 12 weeks, 9 months, or a year. I've been programming Perl for nearly five years and I'm still learning.
How true. Especially for a language like Perl which allows you to do virtually anything in an uncountable no. of ways. Probably only Larry Wall and a handful few elites can really claim they are done learning. Not ever for the rest of us here.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SQL query output convert to HTML & send as email body

Hi , I have a sql query in the unix script ,whose output is shown below.I want to convert this output to HTML table format & send email from unix with this table as email body. p_id src_system amount 1 A 100 2 B 200 3 C ... (3 Replies)
Discussion started by: jagadeeshn04
3 Replies

2. Shell Programming and Scripting

Parsing the http post request

Hi, I am trying to write a shell script to parse the post request data that it received to a xml file. Below is the post request data that script is receiving. -----------------------------7dd2339190c8e Content-Disposition: form-data; name="param1" 1... (2 Replies)
Discussion started by: jdp
2 Replies

3. Shell Programming and Scripting

Help in parsing XML output file in perl.

Hi I have an XML output like : <?xml version="1.0" encoding="ISO-8859-1" ?> - <envelope> - <body> - <outputGetUsageSummary> - <usgSumm rerateDone="5"> - <usageAccum accumId="269" accumCaptn="VD_DP_AR" inclUnits="9999999.00" inclUnitsUsed="0.00" shared="false" pooled="false"... (7 Replies)
Discussion started by: rkrish
7 Replies

4. Shell Programming and Scripting

http request

I am running a website but I still have problems with the "service temporarily unavailable error". I want to make a simple check if the website is up and running. Does anybody has an idea how to do it? (the site is password protected, so you have to add a user and pwd before logging in). ... (2 Replies)
Discussion started by: jurgen
2 Replies

5. Shell Programming and Scripting

Capture query returned values in file.

Hi All, I am connecting to Oracle DB from UNIX script. Want to capture all dates between start date and end date and store them in file. Once this is done, want to read dates one by one. How to achive this in UNIX and Oracle? Please let me know if you have any idea on the same. Thanks and... (4 Replies)
Discussion started by: Nagaraja Akkiva
4 Replies

6. Shell Programming and Scripting

Awk & sed query for output

Hello, I have a file. its content are like below. mdn:87439842 imsi:23082038203 Ctime:12082010 01:20:10 mdn:9324783783 imsi:402349823322 Ctime: 12072010 01:20:10 mdn:87439842 imsi:23082038203 Ctime: 23072010 01:20:10 mdn:87439842 imsi:23082038203 Ctime:18072010 01:20:10 mdn:87439842... (3 Replies)
Discussion started by: Sanket11
3 Replies

7. Shell Programming and Scripting

How to send XML data using HTTP Post Request

How to hit HTTP Post Request along with sending XML data to a Remote server through command line utility like wget (or anything else). (0 Replies)
Discussion started by: sandeep reddy
0 Replies

8. UNIX for Dummies Questions & Answers

HTTP request

Can anybody tell about http request processing in shell script..? (3 Replies)
Discussion started by: noufal
3 Replies

9. Shell Programming and Scripting

Http request in Linux

Hi, i need a guide how to write a script which i can do a http request. Let say the request look like below; http://www.test.com?txid=1&type=service&server=linux I have a list of "txid" (in *.txt) and need to run all "txid" acordingly. So that mean, every transaction i have to refer "txid"... (7 Replies)
Discussion started by: malaysoul
7 Replies

10. SCO

XML Import & HTTP Post

this may be very basic to some but all new to me I have an application running on SCO Unix server which issues an HTTP Post request to a server with the results being returned in I.E browser window in XML format I need to import these results into my customers application and dont know how to... (1 Reply)
Discussion started by: ccarcher
1 Replies
Login or Register to Ask a Question