Since HTML is very similar to XML, you may use an xml tool to parse your file.
Since your HTML-File is not fully standards compliant, the parser complains about it and the file has either be adapted by hand to be compliant or to be preprocessed prior to the parsing. The <br> is the problematic element. Compliant would be <br/> with a slash within the tag.
So you can do it with an xmlparser like xmlstarlet in three steps:
1. Make the html file compliant by replacing the br-Tags
2. Get the wanted HTML-Element with xmlstarlet
3. suppress unwanted empty lines and leading whitespace in data / xmlstarlet output
RudiC, thanks, that works just great if i have a file with the html code, but i store the html code in variable:
This works, but i store the html code in a variable, not a file: This is my final solution
oh, and also i had to get rid of the semicolons because i had an error sed: 1: "/<p/,/<\/p/ {/<p.*\/p>/ ...": unexpected EOF (pending }'s)
and found that getting rid of the semicolons and using newline instead fixes this error.
Thanks everyone for the help.
--- Post updated at 09:42 PM ---
stomp, i like your solution too, looks very clean unfortunately xmlstarlet is very picky,
in my real life problem it's not just <br>-s that needs to be transformed to be compliant and would be overkill to check and transform the whole html page for xmlstarlet
But glad that you showed me this, i might use it somewhere else.
RudiC, i'm not dropping it, because i need to get other texts out of the html, but for the example sakes, yes that would make it more optimized.
I have 5 more texts that i'm matching and making the output into a csv file.
The html from which i'm parsing is built up very poorly.
Actual code snippet:
so as you can see it has a lot of spaces, and new lines at the end and beginning.
Your solution doesn't deal with those so i came up with this:
Since i need this all in one line or else the csv file will broke (just realized this) had to get rid of the new lines tr -d "\n\r"
I' removing the extra whitespaces at the beginning and end awk '{$1=$1};1'
Also for csv proofing i'm replacing the commas with semicolon because csv will interpret commas as end of column tr ',' ';'
So this makes me wonder if that one sed could do all these by on it's own.
But i'm happy now because this works now.
I want to clean a html file.
I try to remove the script part in the html and remove the rest of tags and empty lines.
The code I try to use is the following:
sed '/<script/,/<\/script>/d' webpage.html | sed -e 's/<*>//g' | sed '/^\s*$/d' > output.txt
However, in this method, I can not... (10 Replies)
Hi Guys
Here is my Input :
<?xml version="1.0" encoding="UTF-8"?>
<xn:MeContext id="01736">
<xn:VsDataContainer id="01736">
<xn:attributes>
<xn:vsDataType>vsDataMeContext</xn:vsDataType>
... (12 Replies)
I want to print from <fruits> to </fruits> tag which have <fruit> as mango. Also i want both <fruits> and </fruits> in output. Please help
eg.
<fruits>
<fruit id="111">mango<fruit>
.
another 20 lines
.
</fruits> (3 Replies)
I have an XML tag like this:
<property name="agent" value="/var/tmp/root/eclipse" />
Is there way using awk that i can get the value from the above tag. So the output should be:
/var/tmp/root/eclipse
Help will be appreciated.
Regards,
Adi (6 Replies)
Guys,
I have a little script that I got of the internet and that I use in Squid to block ads.
I used that script with linux but now i have moved my servers to freebsd. I have a step learning curve there but it is fun: Back to the script issue.
The script used to work i with linux but... (15 Replies)
Hi!
I have a bunch of HTML files, which I want to parse to CSV files. Every page has a table in it, and I need to parse each row into a csv record.
With awk and sed, I managed to put every table row in separate lines. So my file looks like this:
<TR> .... </TR>
<TR> .... </TR>
...One... (1 Reply)
hi all,
i have a html file something similar to this.
<tr class="evenrow">
<td class="data">added</td><td class="data">xyz@abc.com</td>
<td class="data">filename.sql</td><td class="modifications-data">08/25/2009 07:58:40</td><td class="data">Added TK prof script</td>
</tr>
<tr... (1 Reply)
Hai friends
I have a small doubt..
how can we use html tag in shell scripting
code :
echo "<html>"
echo "<body>"
echo " welcome to peace world "
echo "</body>"
echo "</html>"
output displayed like this:
<html>
<body>
welcome to peace world
</body>
</html> (5 Replies)
I have a html file called myfile. If I simply put "cat myfile.html" in UNIX, it shows all the html tags like <a href=r/26><img src="http://www>. But I want to extract only text part.
Same problem happens in "type" command in MS-DOS.
I know you can do it by opening it in Internet Explorer,... (4 Replies)