Pattern matching extracting urls from rss, shell scripts


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Pattern matching extracting urls from rss, shell scripts
# 1  
Old 04-30-2009
Pattern matching extracting urls from rss, shell scripts

Hi all, how could i do ?

I have a Rss file, i want to extract only the Urls (many) matching http://www.xxx.com/trailers/ from that file and copy into another file.


like

"
<pubDate>Wed, 29 Apr 2009 00:00:00 PST</pubDate>
<content:encoded><![CDATA[<table><tr valign="top"><td width="67"><a href="Apple - Movie Trailers - The Hangover"><img src="http://images.apple.com/moviesxml/s/wb/posters/thehangover_m200904271417.jpg" width="65" height="97" border="0"></a></td><td> &nbsp; </td><td><a href="Apple - Movie Trailers - The Hangover/"><span style="font-size: 16px; font-weight: 900; text-decoration: underline;">The Hangover - Trailer 2</span></a><br /><span style="font-size: 12px;">Two days before his wedding, Doug and his three friends drive to Las Vegas for a blow-out bachelor party they&#x2019;ll never forget. But, in fact, when the three groomsmen wake up the ustin Bartha</span></td></tr></table>]]></content:encoded> .....

all made with bash script file.

thanks 4 help !

Smilie
# 2  
Old 04-30-2009
Code:
cat << EOF |
<pubDate>Wed, 29 Apr 2009 00:00:00 PST</pubDate>
<content:encoded><![CDATA[<table><tr valign="top"><td width="67"><a href="Apple - Movie Trailers - The Hangover"><
img src="http://images.apple.com/moviesxml/s/wb/posters/thehangover_m200904271417.jpg" width="65" height="97" bord
er="0"></a></td><td> &nbsp; </td><td><a href="Apple - Movie Trailers - The Hangover/"><span style="font-size: 16px
; font-weight: 900; text-decoration: underline;">The Hangover - Trailer 2</span></a><bill_run_id /><span style="fo
nt-size: 12px;">Two days before his wedding, Doug and his three friends drive to Las Vegas for a blow-out bachelor
 party they&#x2019;ll never forget. But, in fact, when the three groomsmen wake up the ustin Bartha</span></td></t
r></table>]]></content:encoded> .....
EOF
tr '<' '\012' |
tr '>' '\012' |
grep '^a href' |
sed -e 's/a href=.//' \
    -e 's/.$//'


Last edited by quirkasaurus; 04-30-2009 at 02:49 PM.. Reason: a href was off.
# 3  
Old 04-30-2009
Maybe something like this:

Code:
perl -ne '{ while (/.*?(http:\/\/www.*?.com\/trailers\/).*?/gi) { print $1,"\n"; } }' <your_filename>

HTH,
tyler_durden

______________________________________________
"Only after disaster can we be resurrected."
# 4  
Old 04-30-2009
Quote:
Originally Posted by durden_tyler
Maybe something like this:

Code:
perl -ne '{ while (/.*?(http:\/\/www.*?.com\/trailers\/).*?/gi) { print $1,"\n"; } }' <your_filename>

HTH,
tyler_durden

______________________________________________
"Only after disaster can we be resurrected."
thanks lot, that fine, works well

Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting urls from curl output

Hello. I use curl to fetch a website, then, I want to extract the URLs from this curls output. I tried both sed and grep, but couldnt figure it out. Ive tried : sed -n 's/href="\(*\).*/\1/p' results.txt and grep -o grep -o '<a href="http://*.*.*/*">' results.txt. What pattern... (6 Replies)
Discussion started by: jozo95
6 Replies

2. UNIX for Dummies Questions & Answers

Extracting sub-string matching the pattern.

Hi, I have a string looks like the following: USERS 32767.9844 UNDOTBS1 32767.9844 SYSAUX 32767.9844 SYSTEM 32767.9844 EMS 8192 EMS 8192 EMS_INDEXES 4096 EMS_INDEXES 4096 8 rows selected. How do I extract a sub-string to get the expected output as following: EMS 8192 EMS_INDEXES 4096 ... (3 Replies)
Discussion started by: NetBear
3 Replies

3. Shell Programming and Scripting

Pattern Matching and extracting the required fields in Perl

Hi All, I am writing the following Perl Scrip and need your help in Pattern matching : I have the following Shell Script that would read line by line from the file (file_svn) and would inturn calls the Perl Script: #!/bin/bash perl_path="/home/dev/filter"... (2 Replies)
Discussion started by: filter
2 Replies

4. Shell Programming and Scripting

Korn Shell for pattern matching and extracting

Guys, i'm new to shell scripting. Here's what i need. I need a shell script which would read a file containing only 1 line which never changes. File containts - SQL_Mgd_Svc_ELONMCL54496 |EMEA\brookkev, EMEA\fieldgra, EMEA\tidmamar, EMEA\attfiste, EMEA\baldogar, EMEA\clarkia2, EMEA\conwasha,... (9 Replies)
Discussion started by: butterfly20
9 Replies

5. Shell Programming and Scripting

Extracting a string matching a pattern from a line

Hi All, I am pretty new to pattern matching and extraction using shell scripting. Could anyone please help me in extracting the word matching a pattern from a line in bash. Input Sample (can vary between any of the 3 samples below): 1) Adaptec SCSI RAID 5445 2) Adaptec SCSI 5445S RAID 3)... (8 Replies)
Discussion started by: jharish
8 Replies

6. Shell Programming and Scripting

Extracting the strings matching a pattern from a word

Hi All , I need to extract the strings that are matching with the pattern : CUST.<AnyStringOfAnyLength>.<AnyStringOfAnyLength> from a file and then write all these string into another file. e.g. If a file SOURCE contains following lines : IF(CUST.ABCD.EFGH==1) THEN CUST.ABCD.EFGH =... (7 Replies)
Discussion started by: swapnil.nawale
7 Replies

7. Shell Programming and Scripting

shell script pattern matching

Hi, I need to create a shell script through which i need to populate email addresses in email columns of database table in mysql. Let say if email contains yahoo, hotmail, gtalk than email addresses need to move in their respective columns. # !/bin/sh yim="example@yahoo.com"... (3 Replies)
Discussion started by: mirfan
3 Replies

8. Shell Programming and Scripting

help extracting a matching pattern and next lines of match

Hi there, i'm having some problems just making an awk script (i've tried this way, but other way can be posible for sure), for the next file file.txt <register> <createProfile> <result>0</result> <description><!]></description> <msisdn>34661461174</msisdn> <inputOmvID>1</inputOmvID>... (6 Replies)
Discussion started by: vicious
6 Replies

9. Shell Programming and Scripting

Problem extracting just a part of a matching pattern

Hello everyone, this is my first post so please give me a hand. I apologize for my English, I'll try to be clear with my request. I need to write a script (Bash) which finds all the variables defined in the file .h of the folder and then writes the name of the files .c where these variables are... (1 Reply)
Discussion started by: paxilpaz
1 Replies

10. Shell Programming and Scripting

shell pattern matching

Hello Members I am facing a problem regarding pattern matching.please guide me to solve the issue.My requirement is like: There is table in oracle database, in that table contain columns ,inside the column so many files are there. my requirement is that to search a pattern for example: pattern... (5 Replies)
Discussion started by: rakeshforum
5 Replies
Login or Register to Ask a Question