here's a suggestion using pup, a html-parser written in go:
Explanation: Get all p-Elements with div-elements as parents and output the text data of it.
To get rid of the empty lines, I suggest a small sed command afterwards:
Another short demonstration of pup which I shortly used to get the numbers of cases for the coronovirus out of a complex website and into variables(for generating this graph: coronavirus statistics)) with only one combined command:
I have a html file called myfile. If I simply put "cat myfile.html" in UNIX, it shows all the html tags like <a href=r/26><img src="http://www>. But I want to extract only text part.
Same problem happens in "type" command in MS-DOS.
I know you can do it by opening it in Internet Explorer,... (4 Replies)
Hai friends
I have a small doubt..
how can we use html tag in shell scripting
code :
echo "<html>"
echo "<body>"
echo " welcome to peace world "
echo "</body>"
echo "</html>"
output displayed like this:
<html>
<body>
welcome to peace world
</body>
</html> (5 Replies)
hi all,
i have a html file something similar to this.
<tr class="evenrow">
<td class="data">added</td><td class="data">xyz@abc.com</td>
<td class="data">filename.sql</td><td class="modifications-data">08/25/2009 07:58:40</td><td class="data">Added TK prof script</td>
</tr>
<tr... (1 Reply)
Hi!
I have a bunch of HTML files, which I want to parse to CSV files. Every page has a table in it, and I need to parse each row into a csv record.
With awk and sed, I managed to put every table row in separate lines. So my file looks like this:
<TR> .... </TR>
<TR> .... </TR>
...One... (1 Reply)
Guys,
I have a little script that I got of the internet and that I use in Squid to block ads.
I used that script with linux but now i have moved my servers to freebsd. I have a step learning curve there but it is fun: Back to the script issue.
The script used to work i with linux but... (15 Replies)
I have an XML tag like this:
<property name="agent" value="/var/tmp/root/eclipse" />
Is there way using awk that i can get the value from the above tag. So the output should be:
/var/tmp/root/eclipse
Help will be appreciated.
Regards,
Adi (6 Replies)
I want to print from <fruits> to </fruits> tag which have <fruit> as mango. Also i want both <fruits> and </fruits> in output. Please help
eg.
<fruits>
<fruit id="111">mango<fruit>
.
another 20 lines
.
</fruits> (3 Replies)
Hi Guys
Here is my Input :
<?xml version="1.0" encoding="UTF-8"?>
<xn:MeContext id="01736">
<xn:VsDataContainer id="01736">
<xn:attributes>
<xn:vsDataType>vsDataMeContext</xn:vsDataType>
... (12 Replies)
I want to clean a html file.
I try to remove the script part in the html and remove the rest of tags and empty lines.
The code I try to use is the following:
sed '/<script/,/<\/script>/d' webpage.html | sed -e 's/<*>//g' | sed '/^\s*$/d' > output.txt
However, in this method, I can not... (10 Replies)
Discussion started by: YuhuiFeng
10 Replies
LEARN ABOUT DEBIAN
hxnormalize
HXNORMALIZE(1) HTML-XML-utils HXNORMALIZE(1)NAME
hxnormalize - pretty-print an HTML file
SYNOPSIS
hxnormalize [ -x ] [ -e ] [ -d ] [ -s ] [ -L ] [ -i indent ] [ -l line-length ] [ -c commentmagic ] [ file-or-URL ]
DESCRIPTION
The hxnormalize command pretty-prints an HTML file, and also tries to fix small errors. The output is the same HTML, but with a maximum
line length and with optional indentation to indicate the nesting level of each line.
OPTIONS
The following options are supported:
-x Use XML conventions: empty elements are written with a slash at the end: <IMG />. Implies -e.
-e Always insert endtags, even if HTML does not require them (for example: </p> and </li>).
-d Omit the DOCTYPE from the output.
-i indent Set the number of spaces to indent each nesting level. Default is 2. Not all elements cause an indent. In general, elements that
can occur in a block environment are started on a new line and cause an indent, but inline elements, such as EM and SPAN do not
cause an indent.
-l line-length
Sets the maximum length of lines. hxnormalize will wrap lines so that all lines are as long as possible, but no longer than this
length. Default is 72. Words that are longer than the line length will not be broken, and will extend past this length. A
content of the STYLE, SCRIPT and PRE elements will not be line-wrapped.
-s Omit <span> tags that don't have any attributes.
-L Remove redundant "lang" and "xml:lang" attributes. (I.e., those whose value is the same as the language inherited from the parent
element.)
-c commentmagic
Comments are normally placed right after the preceding text. That is usually correct for short comments, but some comments are
meant to be on a separate line. commentmagic is a string and when that string occurs inside a comment, hxnormalize will output
an empty line before that comment. E.g. -c "====" can be used to put all comments that contain "====" on a separate line, pre-
ceded by an empty line. By default, no comments are treated that way.
OPERANDS
The following operand is supported:
file-or-URL
The name or URL of an HTML file. If absent, standard input is read instead.
EXIT STATUS
The following exit values are returned:
0 Successful completion.
> 0 An error occurred in the parsing of the HTML file. hxnormalize will try to correct the error and produce output anyway.
ENVIRONMENT
To use a proxy to retrieve remote files, set the environment variables http_proxy and ftp_proxy. E.g., http_proxy="http://localhost:8080/"
BUGS
The error recovery for incorrect HTML is primitive.
hxnormalize will not omit an endtag if the white space after it could possibly be significant. E.g., it will not remove the first </p> from
"<div><p>text</p> <p>text</p></div>".
hxnormalize can currently only retrieve remote files over HTTP. It doesn't handle password-protected files, nor files whose content depends
on HTTP "cookies."
SEE ALSO asc2xml(1), xml2asc(1), UTF-8 (RFC 2279)
6.x 10 Jul 2011 HXNORMALIZE(1)