Sponsored Content
Top Forums Shell Programming and Scripting Trying to extract domain and tld from list of urls. Post 302641875 by chamb1 on Wednesday 16th of May 2012 03:23:52 PM
Old 05-16-2012
Trying to extract domain and tld from list of urls.

I have done a fair amount of searching the threads, but I have not been able to cobble together a solution to my challenge. What I am trying to do is to line edit a file that will leave behind only the domain and tld of a long list of urls. The list looks something like this:
Code:
www.google.com
ja.wikipedia.org
bbc.co.uk
fr-fr.facebook.com

and I would like to end up with:
Code:
google.com
wikipedia.org
bbc.co.uk
facebook.com

I prefer bash, but am learning ruby and perl....though not very good at them yet. I have used ruby's URI function to extract the input links above...is there another ruby function I am overlooking for domain.tld?

Thanks!
 

10 More Discussions You Might Find Interesting

1. Email Antispam Techniques and Email Filtering

Sendmail Access DB TLD Blocking ....

Now this is a bit tricky, but works great if you can decide which Top Level Domains or TLDs you want to receive mail We are getting so much spam from countries we never receive useful mail, I've been experimenting with blocking entire TLDs using sendmail access_db as an antispam technique. ... (0 Replies)
Discussion started by: Neo
0 Replies

2. Solaris

List of Hostname under NIS Domain

How do I find a list of hosts under a domainname on a NIS+ I did check nisls command , I could not find any ??? (5 Replies)
Discussion started by: sriram003
5 Replies

3. Shell Programming and Scripting

Rsync to an external list of URLs

I'm going to have a text file formatted something like this: some_name http://www.someurl.com/ another_name http://www.anotherurl.com/ third_name http://www.thirdurl.com/ I need to write a script that can rsync from a file path I'll set, to each URL in the list. Any ideas? (8 Replies)
Discussion started by: ibsen
8 Replies

4. Shell Programming and Scripting

finding and removing patterns in a large list of urls

I have a list of urls for example: Google Google Base Yahoo! Yahoo! Yahoo! Video - It's On Google The problem is that Google and Google are duplicates as are Yahoo! and Yahoo!. I'm needing to find these conical www duplicates and append the text "DUP#" in from of both Google and... (3 Replies)
Discussion started by: totus
3 Replies

5. Windows & DOS: Issues & Discussions

How to: Linux BOX in Windows Domain (w/out joining the domain)

Dear Expert, i have linux box that is running in the windows domain, BUT did not being a member of the domain. as I am not the System Administrator so I have no control on the server in the network, such as modify dns entry , add the linux box in AD and domain record and so on that relevant. ... (2 Replies)
Discussion started by: regmaster
2 Replies

6. Shell Programming and Scripting

Extract URLs from HTML code using sed

Hello, i try to extract urls from google-search-results, but i have problem with sed filtering of html-code. what i wont is just list of urls thay apears between ........<p><a href=" and next following " in html code. here is my code, i use wget and pipelines to filtering. wget works, but... (13 Replies)
Discussion started by: L0rd
13 Replies

7. Shell Programming and Scripting

Extract urls from index.html downloaded using wget

Hi, I need to basically get a list of all the tarballs located at uri I am currently doing a wget on urito get the index.html page Now this index page contains the list of uris that I want to use in my bash script. can someone please guide me ,. I am new to Linux and shell scripting. ... (5 Replies)
Discussion started by: mnanavati
5 Replies

8. What is on Your Mind?

Tld.subdomain.name.subname

Way back in the early dawn of the 'net, there were two competing notations for specifying a FQDN, the familiar name.subdomain.domain.tld (such as news.bbc.co.uk) and the reversed tld.domain.subdomain.name (uk.co.bbc.news). And if memory serves, only the UK used the latter style of FQDN for a period... (0 Replies)
Discussion started by: derekludwig
0 Replies

9. Red Hat

List domain groups

Hi Need to list all gid for particular domain user. Actually in database getting error like one of the gid that user belongs is invalid. please suggest. thanks Paul (1 Reply)
Discussion started by: Mathew_paul
1 Replies

10. Solaris

How to list physical CPU on primary domain?

How to list physical CPU on primary domain? Sparc SPARC T5-4 psrinfo -p 1 in ILOM I see Processors: 4 / 4 (2 Replies)
Discussion started by: thomasj
2 Replies
RUBY-SWITCH(1)															    RUBY-SWITCH(1)

NAME
ruby-switch - switch between different Ruby interpreters USAGE
ruby-switch --list ruby-switch --check ruby-switch --set RUBYVERSION ruby-switch --auto DESCRIPTION
ruby-switch can be used to easily switch to different Ruby interpreters as the default system-wide interpreter for your Debian system. When run with --list, all supported Ruby interpreters are listed. When --check is passed, ruby-switch will check which Ruby interpreter is currently being used. If the settings are inconsistent -- e.g. `ruby` is Ruby 1.8 and `gem` is using Ruby 1.9.1, ruby-switch will issue a big warning. When --set RUBYINTERPRETER is used ruby-switch will switch your system to the corresponding Ruby interpreter. This includes, for example, the default implementations for the following programs: ruby, gem, irb, erb, testrb, rdoc, ri. ruby-switch --set auto will make your system use the default Ruby interpreter currently suggested by Debian. OPTIONS
-h, --help Displays the help and exits. A NOTE ON RUBY 1.9.x Ruby uses two parallel versioning schemes: the `Ruby library compatibility version' (1.9.1 at the time of writing this), which is similar to a library SONAME, and the `Ruby version' (1.9.3 is about to be released at the time of writing). Ruby packages in Debian are named using the Ruby library compatibility version, which is sometimes confusing for users who do not follow Ruby development closely. ruby-switch also uses the Ruby library compatibility version, so specifying `ruby1.9.1' might give you Ruby with version 1.9.2, or with version 1.9.3, depending on the current Ruby version of the `ruby1.9.1' package. COPYRIGHT AND AUTHORS
Copyright (c) 2011, Antonio Terceiro <terceiro@debian.org> This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>. 2011-11-20 RUBY-SWITCH(1)
All times are GMT -4. The time now is 12:38 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy