parsing a webpage - perl lwp


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting parsing a webpage - perl lwp
# 1  
Old 09-28-2012
parsing a webpage - perl lwp

I am requesting for the text parsing section below. Any helps are highly appreciated.

Code:
<tr valign="top"><td nowrap>Source name</td>
<td style="text-align: justify">Sample Name<br></td>

I want Sample Name from above.

In the same file, I have to search for another pattern like this

Code:
<td><a href="https://www.unix.com/sra?term=SRX12345">SRX12345</a></td>

I want SRX12345 from this pattern.

Now, my final output will be

Code:
SRX12345 Sample Name

---------- Post updated 09-28-12 at 08:42 AM ---------- Previous update was 09-27-12 at 03:03 PM ----------

Hi Friends,

I worked on this task and reached till this point.

Could someone please enhance it?

For the first searching, I used this

Code:
cat input | awk -F'>' '/nowrap>Source/ {getline; print $2}'

The output was

Code:
Sample Name<br

For the second pattern, I wrote this

Code:
cat input |awk '/^<td><a href=/'| grep -o 'http://unix.com/sra?term=[^"]*'| awk -F'=' '{print $2}'

and the output was

Code:
SRX12345

But, I would like to join both of them together and the expected final output is

Code:
Sample Name SRX12345

I can't use join or other awk scripts because, I am running them in two separate instances and the order is changing. I have more than 500 search patterns to search this way and I want both of them together in two columns.

Last edited by jacobs.smith; 09-27-2012 at 04:04 PM.. Reason: missed url
# 2  
Old 09-28-2012
try something like this...

Code:
awk -F "[<>]" '/nowrap>Source/ {getline; printf $(NF-4)}
/<td><a href=/{print $(NF-4)}'  file

This User Gave Thanks to pamu For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl Script - Sort Columns on webpage

foreach my $ds (sort { $a->{books} cmp $b->{books} || (($a->{books} eq "A") ? ($a->{hammers} cmp $b->{hammers}) : (($a->{shoes} <=> $b->{shoes}) || ($a->{hammers}... (1 Reply)
Discussion started by: scj2012
1 Replies

2. Shell Programming and Scripting

How to select a particular field from a drop-down menu in a webpage using perl script?

Hi Team, I have a requirement to login into URL using username and password , then I have to select a "particular name" from drop-down menu and then Read the values user records etc.... using perl. Is it possible to do in perl script ? (OR) Can you please let me know which scripting... (1 Reply)
Discussion started by: latika
1 Replies

3. Shell Programming and Scripting

Navigating a WebPage with Perl

Hi All Below is Code, It opens a link from which it ask a login name and password, the script enter the login name and password and navigate to next page.. In the next page there is a drop down box from which i have to select a value, I have written the code but it gives error #!/usr/bin/perl... (3 Replies)
Discussion started by: parthmittal2007
3 Replies

4. Shell Programming and Scripting

perl lwp find word and print next word :)

hi all, I'm new there, I'm just playing with perl and lwp and I just successfully created a script for log in to a web site with post. I have a response but I would like to have something like this: I have in my response lines like: <div class="sender">mimi020</div> <some html code.....>... (3 Replies)
Discussion started by: vogueestylee
3 Replies

5. Shell Programming and Scripting

open a webpage through perl script

Hi All Can anyone please help me how to open a webpage suppose(Google) with the help of perl script and refresh it after every 5 minutes. (0 Replies)
Discussion started by: parthmittal2007
0 Replies

6. Shell Programming and Scripting

Perl LWP user authentication

Hello all.. i am new to perl scripting.. i wanted to parse a text file, encode the parsed text and attach in url.. please point me to right resources if you know any..This is my major problem. Now i try to get a url running and save it in a text file using LWP module in perl, I used following... (0 Replies)
Discussion started by: empyrean
0 Replies

7. Shell Programming and Scripting

Login using perl LWP module

Hi, Could some one tell me how to login to any web site and get that page using perl LWP. I heard that we can login to the site using LWP. I dont want to use WWW:Mechanize as I dont have that module installed on the server. Appreciate your early response. Thanks... (8 Replies)
Discussion started by: Anjan1
8 Replies

8. Shell Programming and Scripting

Perl parsing compared to Ksh parsing

#! /usr/local/bin/perl -w $ip = "$ARGV"; $rw = "$ARGV"; $snmpg = "/usr/local/bin/snmpbulkget -v2c -Cn1 -Cn2 -Os -c $rw"; $snmpw = "/usr/local/bin/snmpwalk -Os -c $rw"; $syst=`$snmpg $ip system sysName sysObjectID`; sysDescr.0 = STRING: Cisco Internetwork Operating System Software... (1 Reply)
Discussion started by: popeye
1 Replies

9. Shell Programming and Scripting

Help with perl script to search webpage

i have to point out that i'm a avid shell script lover and i have next to zero interest in perl which is why i dont know the first thing about it. however, just need to do this one thing on it.. i need to do something in perl but dont know how. I have a list of email addresses in the following... (5 Replies)
Discussion started by: Terrible
5 Replies

10. Shell Programming and Scripting

Help needed in Perl LWP module

Hi, I've a issue in writing a perl script that will automatically monitor the site availability. There are two different cookies are set in two consecutive flows to a URL and this second cookie has to be passed to the third step which actually gives permission to access based upon the cookie. ... (1 Reply)
Discussion started by: dayanandra
1 Replies
Login or Register to Ask a Question