Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Get only domain from url file bind
Top Forums Shell Programming and Scripting Get only domain from url file bind Post 302959584 by RudiC on Wednesday 4th of November 2015 05:15:48 PM
Old 11-04-2015
You might want to give this a try:
Code:
awk '
/^\/\/|^ *$/    {next
                }

FNR!=NR         {for (f in FIVE)  if ($0 ~ "\." f "$")  {print $(NF-5), $(NF-4), $(NF-3), $(NF-2), $(NF-1), $NF; next}
                 for (f in FOUR)  if ($0 ~ "\." f "$")  {print $(NF-4), $(NF-3), $(NF-2), $(NF-1), $NF ; next}
                 for (t in THREE) if ($0 ~ "\." t "$")  {print $(NF-3), $(NF-2), $(NF-1), $NF; next}
                 for (t in TWO)   if ($0 ~ "\." t "$")  {print $(NF-2), $(NF-1), $NF; next}
                 for (o in ONE)   if ($0 ~ "\." o "$")  {print $(NF-1), $NF; next}
                 next
                }

/^\*/           {next}

NF==5           {FIVE[$0]}
NF==4           {FOUR[$0]}
NF==3           {THREE[$0]}
NF==2           {TWO[$0]}  
NF==1           {ONE[$0]}  
' FS="." OFS="." public_suffix_list.dat rawfile
yahoo.com
doubleclick.net
prq.to
akamaihd.net
apple.com
glistockisti.it
ad-x.co.uk
edgekey.net
bbci.co.uk
google.co.nz
bbc.co.uk

The wildcard issue at begin-of-line hasn't been solved yet; and some optimization might come in handy...
Last edited by RudiC; 11-04-2015 at 06:29 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

url calling and parameter passing to url in script

Hi all, I need to write a unix script in which need to call a url. Then need to pass parameters to that url. please help. Regards, gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies

2. Shell Programming and Scripting

url calling and parameter passing to url in script

Hi all, I need to write a unix script in which need to call a url. Then need to pass parameters to that url. please help. Regards, gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies

3. UNIX for Dummies Questions & Answers

ReDirecting a URL to another URL - Linux

Hello, I need to redirect an existing URL, how can i do that? There's a current web address to a GUI that I have to redirect to another webaddress. Does anyone know how to do this? This is on Unix boxes Linux. example: https://m45.testing.address.net/host.php make it so the... (3 Replies)
Discussion started by: SkySmart
3 Replies

4. Windows & DOS: Issues & Discussions

How to: Linux BOX in Windows Domain (w/out joining the domain)

Dear Expert, i have linux box that is running in the windows domain, BUT did not being a member of the domain. as I am not the System Administrator so I have no control on the server in the network, such as modify dns entry , add the linux box in AD and domain record and so on that relevant. ... (2 Replies)
Discussion started by: regmaster
2 Replies

5. Web Development

Regex to rewrite URL to another URL based on HTTP_HOST?

I am trying to find a way to test some code, but I need to rewrite a specific URL only from a specific HTTP_HOST The call goes out to http://SUB.DOMAIN.COM/showAssignment/7bde10b45efdd7a97629ef2fe01f7303/jsmodule/Nevow.Athena The ID in the middle is always random due to the cookie. I... (5 Replies)
Discussion started by: EXT3FSCK
5 Replies

6. UNIX for Dummies Questions & Answers

Awk: print all URL addresses between iframe tags without repeating an already printed URL

Here is what I have so far: find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}' Here is an example content of a PHP or HTM(HTML) file: <iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies

7. Shell Programming and Scripting

Hit multiple URL from a text file and store result in other test file

Hi, I have a problem where i have to hit multiple URL that are stored in a text file (input.txt) and save their output in different text file (output.txt) somewhat like : cat input.txt http://192.168.21.20:8080/PPUPS/international?NUmber=917875446856... (3 Replies)
Discussion started by: mukulverma2408
3 Replies

8. Shell Programming and Scripting

Reading URL using Mechanize and dump all the contents of the URL to a file

Hello, Am very new to perl , please help me here !! I need help in reading a URL from command line using PERL:: Mechanize and needs all the contents from the URL to get into a file. below is the script which i have written so far , #!/usr/bin/perl use LWP::UserAgent; use... (2 Replies)
Discussion started by: scott_cog
2 Replies

9. UNIX for Dummies Questions & Answers

Putting the colon infront of the URL domain

I have a file like this: http://hello.com www.examplecom computer Company I wanted to keep dot (.) infront of com. to make the file like this http://hello.com www.example.com computer Company I applied this expression sed -r 's/com/.com/g'but what I get is: http://hello.com ... (4 Replies)
Discussion started by: csim_mohan
4 Replies

10. UNIX for Dummies Questions & Answers

Extracting URL with domain

I have a file like this: http://article.wn.com/view/2010/11/26/IV_drug_policy_feels_HIV_patients_Red_Cross/ http://aidsjournal.com/,www.cfpa.org.cn/page1/page2 , www.youtube.com http://seattletimes.nwsource.com/html/jerrybrewer/2013517803_brewer25.html... (1 Reply)
Discussion started by: csim_mohan
1 Replies

LEARN ABOUT DEBIAN

bio::phylonetwork::graphviz

Bio::PhyloNetwork::GraphViz(3pm)			User Contributed Perl Documentation			  Bio::PhyloNetwork::GraphViz(3pm)

NAME

       Bio::PhyloNetwork::GraphViz - Interface between PhyloNetwork and GraphViz

SYNOPSIS

	 use Bio::PhyloNetwork;
	 use Bio::PhyloNetwork::GraphViz;

	 my $net=Bio::PhyloNetwork->new(
	     -eNewick=>'((H1,(H1,(H2,l))),H2)t0; (some long label)H1; ("quoted label")H2;'
	 );

	 my $gv=Bio::PhyloNetwork::GraphViz->new(-net=>$net,-short_labels=>1);

	 foreach my $u ($net->nodes()) {
	     print "$u:".$gv->nodePN_to_nodeGV->{$u}."
";
	 }

	 print $gv->as_text;

	 open PS, "> net.ps";
	 print PS $gv->as_ps;
	 close PS;

DESCRIPTION

       This is a module to create GraphViz objects representing phylogenetic networks.

AUTHOR

       Gabriel Cardona, gabriel(dot)cardona(at)uib(dot)es

SEE ALSO

       Bio::PhyloNetwork, GraphViz

APPENDIX

       The rest of the documentation details each of the object methods.

   new
	Title	: new
	Usage	: my $graphv = new Bio::PhyloNetwork::GraphViz();
	Function: Creates a new Bio::PhyloNetwork::GraphViz object
	Returns : Bio::PhyloNetwork::GraphViz
	Args	: -net => Bio::PhyloNetwork object
		  -short_labels => boolean (optional)

       Returns a Bio::PhyloNetwork::GraphViz object, which is an extension of a GraphViz object. The GraphViz object is a representation of the
       phylogenetic network given. The extra information the created object holds is a hash table with keys the nodes of the PhyloNetwork object
       and values the nodes of the GraphViz object. If the optional argument -short_labels=>1 is given, the labels of the nodes in GraphViz are
       shortened to a maximum of 3 letters.

   nodePN_to_nodeGV
	Title	: nodePN_to_nodeGV
	Usage	: my $hashR=$graphv->nodePN_to_nodeGV()
	Function: returns (a reference to) a hash holding the translation between
		  nodes of the Bio::PhyloNetwork object and nodes of the GraphViz
		  object
	Returns : reference to hash
	Args	: none

perl v5.14.2							    2012-03-02					  Bio::PhyloNetwork::GraphViz(3pm)
All times are GMT -4. The time now is 05:23 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy