Sponsored Content
Full Discussion: Extracting URL with domain
Top Forums UNIX for Dummies Questions & Answers Extracting URL with domain Post 302910037 by csim_mohan on Tuesday 22nd of July 2014 06:36:57 AM
Old 07-22-2014
Extracting URL with domain

I have a file like this:
Code:
http://article.wn.com/view/2010/11/26/IV_drug_policy_feels_HIV_patients_Red_Cross/      http://aidsjournal.com/,www.cfpa.org.cn/page1/page2 , www.youtube.com

http://seattletimes.nwsource.com/html/jerrybrewer/2013517803_brewer25.html
http://www.moortowntoday.co.uk/your-moortown/Yorkshire-Evening-Post-First-for.6038672.jp        www.yorkshireeveningpost.co.uk/business/1/

I want to extract the URLs with the domain
Code:
http://article.wn.com        http://aidsjournal.com,www.cfpa.org.cn,www.youtube.com
http://seattletimes.nwsource.com      http://www.moortowntoday.co.uk ,www.yorkshireeveningpost.co.uk

Any suggestion to achieve this.

Last edited by csim_mohan; 07-22-2014 at 07:50 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

extracting domain names out of a text file

I am needing to extract and list domain names out of a very large text file. The text file contains tlds .com .net .org and others as well as third level domains e.g. host1.domain.com and the names are placed within paragraphs of text. Domains do not have a http:// prefix so I'm thinking the... (6 Replies)
Discussion started by: totus
6 Replies

2. Shell Programming and Scripting

Extracting anchor text and its URL from HTML files in BASH

Hi All, I have some HTML files and my requirement is to extract all the anchor text words from the HTML files along with their URLs and store the result in a separate text file separated by space. For example, <a href="/kid/stay_healthy/">Staying Healthy</a> which has /kid/stay_healthy/ as... (3 Replies)
Discussion started by: shoaibjameel123
3 Replies

3. UNIX for Dummies Questions & Answers

Awk: print all URL addresses between iframe tags without repeating an already printed URL

Here is what I have so far: find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}' Here is an example content of a PHP or HTM(HTML) file: <iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies

4. Shell Programming and Scripting

Extracting the file name from the specified URL

Hello Everyone, I am trying to write a shell script(or Perl Script) that would do the following: I have a file that contains the following lines: File: https://ims-svnus.com/dev/DB/trunk/feeds/templates/shell_script.txt -r860... (5 Replies)
Discussion started by: filter
5 Replies

5. Shell Programming and Scripting

Reading URL using Mechanize and dump all the contents of the URL to a file

Hello, Am very new to perl , please help me here !! I need help in reading a URL from command line using PERL:: Mechanize and needs all the contents from the URL to get into a file. below is the script which i have written so far , #!/usr/bin/perl use LWP::UserAgent; use... (2 Replies)
Discussion started by: scott_cog
2 Replies

6. Shell Programming and Scripting

Extracting the column containing URL from a text file

I have the file like this: Timestamp URL Text 1331635241000 http://example.com Peoples footage at www.test.com,http://example4.com 1331635231000 http://example1.net crack the nuts http://example6.com 1331635280000 http://example2.net ... (0 Replies)
Discussion started by: csim_mohan
0 Replies

7. Shell Programming and Scripting

Extracting the column containing URL from a text file

I have the file like this: Timestamp URL Text 1331635241000 http://example.com Peoples footage at www.test.com,http://example4.com 1331635231000 http://example1.net crack the nuts http://example6.com 1331635280000 http://example2.net ... (0 Replies)
Discussion started by: csim_mohan
0 Replies

8. Shell Programming and Scripting

Extracting the column containing URL from a text file

I have the file like this: Timestamp URL Text 1331635241000 http://example.com Peoples footage at www.test.com,http://example4.com 1331635231000 http://example1.net crack the nuts http://example6.com 1331635280000 http://example2.net ... (3 Replies)
Discussion started by: csim_mohan
3 Replies

9. UNIX for Dummies Questions & Answers

Putting the colon infront of the URL domain

I have a file like this: http://hello.com www.examplecom computer Company I wanted to keep dot (.) infront of com. to make the file like this http://hello.com www.example.com computer Company I applied this expression sed -r 's/com/.com/g'but what I get is: http://hello.com ... (4 Replies)
Discussion started by: csim_mohan
4 Replies

10. Shell Programming and Scripting

Get only domain from url file bind

Hello everybody I have been trying to extract the domain name from the bind query log with different options, however always get stuck with domains that end with link .co.uk or .co.nz. I tried the following, however only provides the first level: awk -F"." '{print $(NF-1)"."$NF}' list.txt >... (30 Replies)
Discussion started by: omuhans123
30 Replies
Locale::Codes::Language(3pm)				 Perl Programmers Reference Guide			      Locale::Codes::Language(3pm)

NAME
Locale::Codes::Language - standard codes for language identification SYNOPSIS
use Locale::Codes::Language; $lang = code2language('en'); # $lang gets 'English' $code = language2code('French'); # $code gets 'fr' @codes = all_language_codes(); @names = all_language_names(); DESCRIPTION
The "Locale::Codes::Language" module provides access to standard codes used for identifying languages, such as those as defined in ISO 639. Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 639 two- letter codes will be used. SUPPORTED CODE SETS
There are several different code sets you can use for identifying languages. A code set may be specified using either a name, or a constant that is automatically exported by this module. For example, the two are equivalent: $lang = code2language('en','alpha-2'); $lang = code2language('en',LOCALE_CODE_ALPHA_2); The codesets currently supported are: alpha-2, LOCALE_LANG_ALPHA_2 This is the set of two-letter (lowercase) codes from ISO 639-1, such as 'he' for Hebrew. It also includes additions to this set included in the IANA language registry. This is the default code set. alpha-3, LOCALE_LANG_ALPHA_3 This is the set of three-letter (lowercase) bibliographic codes from ISO 639-2 and 639-5, such as 'heb' for Hebrew. It also includes additions to this set included in the IANA language registry. term, LOCALE_LANG_TERM This is the set of three-letter (lowercase) terminologic codes from ISO 639. ROUTINES
code2language ( CODE [,CODESET] ) language2code ( NAME [,CODESET] ) language_code2code ( CODE ,CODESET ,CODESET2 ) all_language_codes ( [CODESET] ) all_language_names ( [CODESET] ) Locale::Codes::Language::rename_language ( CODE ,NEW_NAME [,CODESET] ) Locale::Codes::Language::add_language ( CODE ,NAME [,CODESET] ) Locale::Codes::Language::delete_language ( CODE [,CODESET] ) Locale::Codes::Language::add_language_alias ( NAME ,NEW_NAME ) Locale::Codes::Language::delete_language_alias ( NAME ) Locale::Codes::Language::rename_language_code ( CODE ,NEW_CODE [,CODESET] ) Locale::Codes::Language::add_language_code_alias ( CODE ,NEW_CODE [,CODESET] ) Locale::Codes::Language::delete_language_code_alias ( CODE [,CODESET] ) These routines are all documented in the Locale::Codes::API man page. SEE ALSO
Locale::Codes The Locale-Codes distribution. Locale::Codes::API The list of functions supported by this module. http://www.loc.gov/standards/iso639-2/ Source of the ISO 639-2 codes. http://www.loc.gov/standards/iso639-5/ Source of the ISO 639-5 codes. http://www.iana.org/assignments/language-subtag-registry The IANA language subtag registry. AUTHOR
See Locale::Codes for full author history. Currently maintained by Sullivan Beck (sbeck@cpan.org). COPYRIGHT
Copyright (c) 1997-2001 Canon Research Centre Europe (CRE). Copyright (c) 2001-2010 Neil Bowers Copyright (c) 2010-2012 Sullivan Beck This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.16.2 2012-10-11 Locale::Codes::Language(3pm)
All times are GMT -4. The time now is 02:48 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy