Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Using SED to print the links present in a file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using SED to print the links present in a file
# 1  
Old 11-01-2010
MySQL Using SED to print the links present in a file
i was trying to print the links present in a file using SED.
Code:
sed -n 's/.*\(\(http\|ftp\|https\):\/\/\([^ ]*\).\(net\|html\)\).*/\1/gp' example

content of example file is
Code:
The URL for my site is: http://mysite.com/mydoc.html.  You
might also enjoy ftp://yoursite.com/index.net for a good https://mysite.com/mydoc.html gg
place to download files.

output
Code:
http://mysite.com/mydoc.html
https://mysite.com/mydoc.html

the ftp link in the second line is not getting printed, correct me if i were wrong..?Smilie
# 2  
Old 11-01-2010
Do you have grep -o ?
Code:
grep -Eo '(http|ftp|https)://[[:alnum:][:punct:]]*[[:alnum:]]' infile

output:
Code:
http://mysite.com/mydoc.html
ftp://yoursite.com/index.net
https://mysite.com/mydoc.html

Last edited by Scrutinizer; 11-01-2010 at 08:20 AM..
This User Gave Thanks to Scrutinizer For This Post:
rbatte1
# 3  
Old 11-01-2010
MySQL Using SED to print the links present in a file
Quote:
Originally Posted by Scrutinizer
Do you have grep -o ?
Code:
grep -Eo '(http|ftp|https)://[[:alnum:][:punct:]]*[[:alnum:]]' infile

output:
Code:
http://mysite.com/mydoc.html
ftp://yoursite.com/index.net
https://mysite.com/mydoc.html

Thanks man, However the output i got is
Code:
http://mysite.com
ftp://yoursite.com
https://mysite.com

i didn't get the full link, but i took your hint and used egrep
Thanks once again man
Code:
egrep -o "(http|ftp|https)://([^ ]*)\.(net|html)" example

output
Code:
http://mysite.com/mydoc.html
ftp://yoursite.com/index.net
https://mysite.com/mydoc.html

Smilie
# 4  
Old 11-01-2010
Glad you found a solution. I changed my suggestion a couple of times before I had the right one. Probably you were testing an earlier version. You might want to try the latest suggestion from my post, since it is a bit more generic.
# 5  
Old 11-01-2010
Hi,

One solution with 'GNU sed':
Code:
$ cat script.sed
/\(http\|https\|ftp\):\/\// { 
  h; 
  s/.*\(http:\/\/\([^ ]*\).\(net\|html\)\).*/\1/Ip; 
  g; 
  s/.*\(https:\/\/\([^ ]*\).\(net\|html\)\).*/\1/Ip; 
  g; 
  s/.*\(ftp:\/\/\([^ ]*\).\(net\|html\)\).*/\1/Ip; 
}

Code:
$ cat infile
The URL for my site is: http://mysite.com/mydoc.html.  You
might also enjoy ftp://yoursite.com/index.net for a good https://mysite.com/mydoc.html gg
place to download files.

Code:
$ sed -nf script.sed infile
http://mysite.com/mydoc.html
https://mysite.com/mydoc.html
ftp://yoursite.com/index.net

Regards,
Birei
# 6  
Old 11-01-2010
MySQL Using SED to print the links present in a file
Quote:
Originally Posted by Scrutinizer
Glad you found a solution. I changed my suggestion a couple of times before I had the right one. Probably you were testing an earlier version. You might want to try the latest suggestion from my post, since it is a bit more generic.
it works Thanks man...Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

sed read contents of file and print value another file

Trying to use sed to insert the contents of a file into the end of each line in another file file1 This is a line Here is another line This is yet another line Here is a fourth line file2 TEXT desired output This is a line TEXT Here is another line TEXT This is yet another... (6 Replies)
Discussion started by: jimmyf
6 Replies

2. Shell Programming and Scripting

Print lines present in first that match second variable. And ones that does not exist in second.

I have multi line input(var1) and reference(var2) variables. How to capture lines not present in var2 but present in var1? How to capture lines present var2 but not in var1? # configuration from server var1=""" Custom JAX-RS Custom Shared Web 2.0 """ # required configuration... (6 Replies)
Discussion started by: kchinnam
6 Replies

3. UNIX for Dummies Questions & Answers

sed Exact Match when Dot is present

I am trying to replace exact word from my text. I know using the angled brackets does the trick. But it is not working when there is a dot in the text. echo "Bottle BottleWater Bottle Can" | sed 's/\<Bottle\>//g' BottleWater CanBut if my data has a dot or hash in it, it replaces all the... (10 Replies)
Discussion started by: grep_me
10 Replies

4. Shell Programming and Scripting

File Comparison: Print Lines not present in another file

Hi, I have fileA.txt like this. B01B02 D0011718 B01B03 D0012540 B01B04 D0006145 B01B05 D0004815 B01B06 D0012069 B01B07 D0004064 B01B08 D0011988 B01B09 D0012071 B01B10 D0005596 B01B11 D0011351 B01B12 D0004814 B01C01 D0011804 I want to compare this against another file (fileB.txt)... (3 Replies)
Discussion started by: genehunter
3 Replies

5. Shell Programming and Scripting

print summary of directory, and group all symbolic links

I am trying to get a summary of filetypes in a directory, but the total count of symbolic links is not working. I am stuck at the results of the file command. I have used the find command to confirm my expectations, but my bash function is not giving the results I want. Here is my function:... (2 Replies)
Discussion started by: AlphaLexman
2 Replies

6. Shell Programming and Scripting

SED multiple links in one line

I'm using sed to pull links from a rather large html file, and so far it's working rather well, but I've noticed that it is skipping some links. On some lines there are multiple links for example: It is completely skipping the first link in these. Here is what I'm using: sed 's/^.*<a... (4 Replies)
Discussion started by: DrMachin
4 Replies

7. Shell Programming and Scripting

read file-print lines in sed

Hello! Im trying to read file contents. Then, print out every line that has "/bens/here" in the file that was read. cat /my/file.now | sed '/bens/here/p' I keep getting the error asking if I need to predeclare sed? What does predeclaring sed mean? Thanks! Ben (2 Replies)
Discussion started by: bigben1220
2 Replies

8. Shell Programming and Scripting

sed script to print a value from txt file

Hello guys, I would appreciate if someone can help me to write a shell script using sed. From a larget text file I need to print a fixed value of a word. In another words whenever it finds that word, it needs to grab the other line containing "dn" and prints its value. For example: dn:... (7 Replies)
Discussion started by: cmontr
7 Replies

9. UNIX for Dummies Questions & Answers

using awk or sed to print output from one file

dear i have one file regarding >abshabja>sdksjbs>sknakna>snajxcls so i want to be output like >abshabja >sjkabjb >sknakna >snajxcls Any using awk or sed will help thanks (2 Replies)
Discussion started by: cdfd123
2 Replies

10. Shell Programming and Scripting

PERL: print if files present

FOR: Windows NT 4 I want perl to read a directory. there is suposed to be two files in the folder ( file1.ini and file2.ini ) and i want perl to print "Files present" or "Files NOT present" to a text document ( report.txt ) how do i do it.? (2 Replies)
Discussion started by: perleo
2 Replies
Login or Register to Ask a Question