Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Retrieve information Text/Word from HTML code using awk/sed

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Retrieve information Text/Word from HTML code using awk/sed
# 1  
Old 04-11-2014
Retrieve information Text/Word from HTML code using awk/sed
awk/sed newbie here. I have a HTML file and from that file and I would like to retrieve a text word.

HTML Code:
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/abc_process.txt>abc</a> NDK Version:  4.0 </li>
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/abc01_process.txt>abc01</a> NDK Version:  4.0 </li>
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/abc045_process.txt>abc045</a> NDK Version:  4.0 </li>
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/cdf_process.txt>cdf</a> NDK Version:  4.0 </li>
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/Manhattan_process.txt>Manhattan</a> NDK Version:  4.0 </li>
For eg. From the 1st line I would like to retrieve abc placed between: txt>abc</a>

I have used the following command but as you can see that number of letters in the word keeps changing abc, abc01, abc045, cdf, Manhattan.
awk -F\/ '{print substr($4,0,3)}' list.html

So this command is getting the output for only the 3 letter word. However I want to extract the same information (abc01, abc045, cdf, Manhattan) from all the lines in the HTML code. Please help.
# 2  
Old 04-11-2014
Code:
awk -F'[<>]' '{ print $7 }' file.html

# 3  
Old 04-11-2014
I Just ran this but it is giving me no output. Just blank lines. This HTML file is having 5 lines and when I run the command you mentioned I am just getting 5 blank lines.
# 4  
Old 04-11-2014
Ok, here is what I got:
Code:
$ cat file.html
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/abc_process.txt>abc</a> NDK Version:  4.0 </li>
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/abc01_process.txt>abc01</a> NDK Version:  4.0 </li>
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/abc045_process.txt>abc045</a> NDK Version:  4.0 </li>
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/cdf_process.txt>cdf</a> NDK Version:  4.0 </li>
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/Manhattan_process.txt>Manhattan</a> NDK Version:  4.0 </li>

Code:
$ awk -F'[<>]' '{ print $7 }' file.html
abc
abc01
abc045
cdf
Manhattan

You can also try:
Code:
sed 's#.*txt>##;s#<.*##' file.html

This User Gave Thanks to Yoda For This Post:
sk2code
# 5  
Old 04-11-2014
Quote:
Originally Posted by sk2code
I Just ran this but it is giving me no output. Just blank lines. This HTML file is having 5 lines and when I run the command you mentioned I am just getting 5 blank lines.
Does the HTML actually look like the data you pasted, or did you pretty it up? Many times when XML/HTML comes up, 5 "lines" is later found to mean tags not necessarily organized into lines at all.
# 6  
Old 04-12-2014
Code:
awk 'BEGIN{FS = "</a>"} {n=split($1, a, ">"); print a[n]}' file

Code:
srinus@ubuntu:~$ cat sam
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/abc_process.txt>abc</a> NDK Version:  4.0 </li>
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/abc01_process.txt>abc01</a> NDK Version:  4.0 </li>
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/abc045_process.txt>abc045</a> NDK Version:  4.0 </li>
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/cdf_process.txt>cdf</a> NDK Version:  4.0 </li>
<font face=arial size=-1><li><a href=/value_for_clients/Tokyo/Manhattan_process.txt>Manhattan</a> NDK Version:  4.0 </li>
srinus@ubuntu:~$ awk 'BEGIN{FS = "</a>"} {n=split($1, a, ">"); print a[n]}' sam
abc
abc01
abc045
cdf
Manhattan
srinus@ubuntu:~$

# 7  
Old 04-14-2014
Yoda and everyone thanks for your help.
The awk command still results in blank output but the following command helped me.

Code:
sed 's#.*txt>##;s#<.*##' file.html

Thank you!!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk/sed HTML extract

I'm extracting text between table tags in HTML <th><a href="/wiki/Buick_LeSabre" title="Buick LeSabre">Buick LeSabre</a></th> using this: awk -F "</*th>" '/<\/*th>/ {print $2}' auto2 > auto3 then this (text between a href): sed -e 's/\(<*>\)//g' auto3 > auto4 How to shorten this into one... (8 Replies)
Discussion started by: p1ne
8 Replies

2. Shell Programming and Scripting

Perl code to retrieve text from website

perl -MLWP::Simple -le '$s=shift;$c=get("http://www.google.com/intl/en/chrome/devices/chromecast/$s/");$c=~/meta content=(.*?)name=\"Remote free\"/msg; print length($1),"\t$1"' ?gclid=CJDg27OdnL0CFcFlOgodFD8A6Q >output.txt output.txt should be: Chromecast works with devices you already own,... (9 Replies)
Discussion started by: cmccabe
9 Replies

3. Shell Programming and Scripting

Extract word from text (sed,awk, etc...)

Hello, I need some help extracting the number after the RBA e.g 15911688 from the below block of text (e.g: grep RBA |sed .......). The code should be valid for blocks if text generated at different times as well and not for the below text only. ... (2 Replies)
Discussion started by: drbiloukos
2 Replies

4. Shell Programming and Scripting

cut, sed, awk too slow to retrieve line - other options?

Hi, I have a script that, basically, has two input files of this type: file1 key1=value1_1_1 key2=value1_2_1 key4=value1_4_1 ... file2 key2=value2_2_1 key2=value2_2_2 key3=value2_3_1 key4=value2_4_1 ... My files are 10k lines big each (approx). The keys are strings that don't... (7 Replies)
Discussion started by: fzd
7 Replies

5. Shell Programming and Scripting

Execute a C program and retrieve information

Hi I have the following script: #!/bin/sh gcc -o program program.c ./program & PID=$! where i execute a C program and i get its pid. I want to retrieve information about this program (e.g memory consumption) using command top. So far i have: top -d 1.0 -p $PID But i dont know how to... (6 Replies)
Discussion started by: nteath
6 Replies

6. Shell Programming and Scripting

sed/awk to retrieve max year in column

I am trying to retrieve that max 'year' in a text file that is delimited by tilde (~). It is the second column and the values may be in Char format (double quoted) and have duplicate values. Please help. (4 Replies)
Discussion started by: CKT_newbie88
4 Replies

7. Shell Programming and Scripting

How to retrieve digital string using sed or awk

Hi, I have filename in the following format: YUENLONG_20070818.DMP HK_20070818_V0.DMP WANCHAI_20070820.DMP KWUNTONG_20070820_V0.DMP How to retrieve only the digital part with sed or awk and return the following format: 20070818 20070818 20070820 20070820 Thanks! Victor (3 Replies)
Discussion started by: victorcheung
3 Replies

8. Shell Programming and Scripting

SED to extract HTML text data, not quite right!

I am attempting to extract weather data from the following website, but for the Victoria area only: Text Forecasts - Environment Canada I use this: sed -n "/Greater Victoria./,/Fraser Valley./p" But that phrasing does not sometimes get it all and think perhaps the website has more... (2 Replies)
Discussion started by: lagagnon
2 Replies

9. UNIX for Dummies Questions & Answers

retrieve lines using sed, grep or awk

Hi, I'm looking for a command to retrieve a block of lines using sed or grep, probably awk if that can do the job. In below example, By searching for words "Third line2" i'm expecting to retrieve the full block starting with 'BEGIN' and ending with 'END' of the search. Example: ... (3 Replies)
Discussion started by: learning_linux
3 Replies

10. Shell Programming and Scripting

How to use sed to remove html tags including text between them

How to use sed to remove html tags including text between them? Example: User <b> rolvak </b> is stupid. It does not using <b>OOP</b>! and should output: User is stupid. It does not using ! Thank you.. (2 Replies)
Discussion started by: alphagon
2 Replies
Login or Register to Ask a Question