Strip subdomains from domains list Post: 302449019

Sponsored Content

Top Forums Shell Programming and Scripting Strip subdomains from domains list Post 302449019 by agama on Friday 27th of August 2010 10:31:19 PM

08-27-2010

Registered User

According to the ISO standard:

Quote:

All ASCII ccTLD identifiers are two letters long, and all two-letter top-level domains are ccTLDs.

Thus, you could assume that any domain ending in a two character code should print the last two subdomain names in addition to the top level domain:

Code:

# assume url is first field; change if needed
awk '
        {
                gsub( "^.*://", "", $1 );      # ditch the http://  ftp:// etc
                n = split( $1, a, "." );
                if( length( a[n] ) == 2 )       # assuming all two character top level domains are country codes
                        printf( "%s.%s.%s\n", a[n-2], a[n-1], a[n] );
                else
                        printf( "%s.%s\n",  a[n-1], a[n] );
        }
' list-file-name

This also takes into account that

Code:

http://foo.com

should not print anything before the double slants. Caution may need to be exercised as some countries have licensed the use of their top level domain name (e.g. .tv and .fm).

agama

View Public Profile for agama

Find all posts by agama

9 More Discussions You Might Find Interesting

1. IP Networking

Setting up subdomains...

Hello, I'd like to set up a subdomain. For instance, me.mydomain.com, instead of mydomain.com/me. I have a dedicated unix server running Apache OS. How do I do this? Any help given will be greatly appreciated and thanks in advance. -Matt

2. UNIX for Dummies Questions & Answers

multiple domains

Hello, I have 3 domains virtually hosted "name based" the first one "domain1.com" has its ServerName entered as domain1.com. this domain will load in a browser by www.domain1.com or simply domain1.com. the next two domains "domain2.com" and "domain3.com" ServerNames are listed as domain2.com and...

3. UNIX for Dummies Questions & Answers

blocking domains

Dear All , Kindly note I have sun solaries 7 . I want to block a domain who keep sending emails to my domain and users . thanks

4. UNIX for Dummies Questions & Answers

creating domains

ok i am setting up dns or going to do it with solaris 9 once u setup the domain what file can u look @ to see if it setup or not

5. UNIX for Dummies Questions & Answers

Automating creation of sub-domains???

Edit, Sorry realised that muxtape.com is probably not creating a sub-domain for every accout but more than likely using a wildcard in the apache config and then using php/asp/java or similar to look at the url and point it in the right direction. So youraccount.muxtape.com maps to...

6. UNIX for Advanced & Expert Users

Different Nameservers for Different Domains?

I have a system that is connected to a private network with its own DNS (call it "privnet."), and is also connected to the Internet on a separate interface. Is it possible to convince this server to query the private nameserver for the private network's domain (e.g. "host foo.privnet."), and the...

7. Shell Programming and Scripting

find out subdomains

I am the new user I want to find out sub domains in the URL using shell script. e.g http://abcd:8380/matcher This is the URL. The suburl is given below http://abcd:8380/matcher/servlet/viewnudatasource http://abcd:8380/matcher/servlet/addnudatasource...

8. UNIX for Dummies Questions & Answers

Sub domains from report

Hi, I have a report containing severals organization's email address. The address contain several sub domains, and i need to pull those out. mail domain ( example.com) .................. The report column contain mail address in this format : john1@sub1.example.com...

9. IP Networking

VLANs and their domains

Hey everyone. I work in a data center, and I'm working on getting my CCNA. Now when I read articles on the idea of VLAN's it makes sense. Especially if you have multiple switches daisy chained in multiple locations. My two main questions though are that most of these examples use PC's as examples...

LEARN ABOUT DEBIAN

net::domain::tld

Net::Domain::TLD(3pm)					User Contributed Perl Documentation				     Net::Domain::TLD(3pm)

NAME

	 Net::Domain::TLD - Work with TLD names

SYNOPSIS

	 use Net::Domain::TLD qw(tlds tld_exists);
	 my @ccTLDs = tlds('cc');
	 print "TLD ok
" if tld_exists('ac','cc');

DESCRIPTION

	 The purpose of this module is to provide user with current list of
	 available top level domain names including new ICANN additions and ccTLDs
	 Currently TLD definitions have been acquired from the following sources:

	 http://www.icann.org/tlds/
	 http://www.dnso.org/constituency/gtld/gtld.html
	 http://www.iana.org/cctld/cctld-whois.htm

PUBLIC METHODS

	 Each public function/method is described here.
	 These are how you should interact with this module.

       "tlds"

	 This routine returns the tlds requested.

	 my @all_tlds = tlds; #array of tlds
	 my $all_tlds = tlds; #hashref of tlds and their descriptions

	 my @cc_tlds = tlds('cc'); #array of just 'cc' type tlds
	 my $cc_tlds = tlds('cc'); #hashref of just 'cc' type tlds and their descriptions

	 Valid types are:
	   cc		      - country code domains
	   gtld_open	      - generic domains that anyone can register
	   gtld_restricted    - generic restricted registration domains
	   new_open	      - recently added generic domains
	   new_restricted     - new restricted registration domains

       "tld_exists"

	 This routine returns true if the given domain exists and false otherwise.

	 die "no such domain" unless tld_exists($tld); #call without tld type
	 die "no such domain" unless tld_exists($tld, 'new_open'); #call with tld type

COPYRIGHT

	 Copyright (c) 2003-2005 Alex Pavlovic, all rights reserved.  This program
	 is free software; you can redistribute it and/or modify it under the same terms
	 as Perl itself.

AUTHORS

	 Alexander Pavlovic <alex.pavlovic@taskforce-1.com>
	 Ricardo SIGNES <rjbs@cpan.org>

perl v5.10.1							    2011-04-18						     Net::Domain::TLD(3pm)