Shell Programming and Scripting

Hi All,

I have some HTML files and my requirement is to extract all the anchor text words from the HTML files along with their URLs and store the result in a separate text file separated by space. For example,

Code:

<a href="/kid/stay_healthy/">Staying Healthy</a>

which has /kid/stay_healthy/ as the URL or path and Staying Healthy as the anchor text.
I want to extract both the above and store in a text file separated by spaces like

Code:

/kid/stay_healthy/ Staying Healthy

New path and new anchor now comes in another line (newline) and so on.

This is what I have tried so far. Got this code from the internet (to be very honest!):

Code:

awk 'BEGIN{
RS="</a>"
IGNORECASE=1
}
{
  for(q=1;q<=NF;q++){
    if ( $q ~ /href/){
      gsub(/.*href=\042/,"",$q)
      gsub(/\042.*/,"",$q)
      print $(q)
    }
  }
}' file1.html

The problem with the above code is that it is not able to extract the anchor text, second it is doing for a single HTML file. For storing the result in a separate file, I can just redirect the output to a text file using >

What is your system?

Since you have bash, I'm guessing you have linux, so can do this:

Code:

grep "<a href" < input.html | sed -r 's#^.*<a href="([^"]+)">([^<]+)</a>.*$#\1\t\2#' > output.txt

It extracts text like /kid/stay_healthy/ Staying Healthy

Oops sorry! yes its Linux

Hi,

I was just wondering is there any way I can also squeeze in the file name where that anchor link occurs in this command?

Code:

grep "<a href" < input.html | sed -r 's#^.*<a href="([^"]+)">([^<]+)</a>.*$#\1\t\2#' > output.txt

For example, if the above anchor text appears in file abc.html, then my output should be like this:

Code:

/kid/stay_healthy/ Staying Healthy abc.html

words separated by spaces.