html tags

dunryc · #1 (**permalink**) 11-28-2007

hi new to the forum so hi every one hope you all well,

Iam attempting to write a bash script at the moment its a scraper/grabber using wget to download webpages related to the users query. that part is no probs when i have the page i need to stipr all the useless (to me) data out of the html source ie :-

Quote:

<html>
test test test
<tag>test ttest </tag>
<new>
this is the data i want to grab between the new tags
</new>
<html>

as you can seen from the above the data i need to grab is from between the new tags these are always on the source what ever the uses query. Can anyone help or point me in the correct direction any help would be greatly appreciated thanks for listening dunryc

porter · #2 (**permalink**) 11-28-2007

Have you considered XMLStarlet Command Line XML Toolkit: Overview

bakunin · #3 (**permalink**) 11-28-2007

Quote:

Originally Posted by dunryc

the data i need to grab is from between the new tags these are always on the source what ever the uses query.

There are two different cases to be considered: the starting and ending tags are on the same line or they are on different lines:

Code:

Example

<new>This is the text to catch</new>

<new>
This is some text
to catch</new>

Both can be matched by simple regular expressions. For each regexp i give the matched portion in blue:

Code:

sed -n 's/.*<new>\(.*\)<\/new>.*/\1/p'

blabla <new>text to match</new> blabla

sed -n '/<new>/,/<\/new>/ {
               s/.*<new>//
               s/<\/new>.*//
               /^$/d
               p
               }'

blabla <new>text
to
match</new> blabla

bakunin

dunryc · #4 (**permalink**) 11-29-2007

thanks for the pointers guys , i did have a look at XMLStarlet to grab the data and it works great but i wanted to use tools that would be present in most distros the commands that bakunin work great once again thanks for the help

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Remove html tags with bash	dejavu88	Shell Programming and Scripting	4	05-22-2008 01:58 PM
How to supplement HTML tags with SED	DocBrewer	Shell Programming and Scripting	3	04-25-2008 09:40 AM
How to remove only html tags inside a file?	btech_raju	Linux	2	11-23-2007 12:25 PM
Automated replacement of HTML Tags	nem_kirk	SUN Solaris	1	11-17-2005 01:24 AM
unsing sed to strip html tags - help	zap	Shell Programming and Scripting	3	04-18-2004 04:03 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode	Rate This Thread:


	Powered by