Extracting the column containing URL from a text file Post: 302909359

Login or Register to Ask a Question and Join Our Community

Sponsored Content

Top Forums Shell Programming and Scripting Extracting the column containing URL from a text file Post 302909359 by csim_mohan on Wednesday 16th of July 2014 10:49:15 AM

Old

07-16-2014

Registered User

Extracting the column containing URL from a text file

I have the file like this:

Code:

Timestamp       URL                    Text                     1331635241000   http://example.com     Peoples footage at www.test.com,http://example4.com 1331635231000   http://example1.net    crack the nuts http://example6.com    1331635280000   http://example2.net    Loving this

Each column is tab separated. I need to extract only the URLs from column 2 and column 3 if in case of the no URLs then leave it empty for example to get the result like this:

Code:

URL                    Text http://example.com     www.test.com,http://example4.com  http://example1.net    http://example6.com http://example2.net

I tried this script

Code:

awk 'BEGIN {FS="\t"} {print $2,$3}' file | grep -oP '(((http|https|ftp|gopher)|mailto)[.:][^ >"\t]*|www\.[-a-z0-9.]+)[^ .,;\t>">\):]'

This script can give me the all URLS in a single column without the header. Any suggestion to resolve this.

csim_mohan

View Public Profile for csim_mohan

Find all posts by csim_mohan

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting anchor text and its URL from HTML files in BASH

Hi All, I have some HTML files and my requirement is to extract all the anchor text words from the HTML files along with their URLs and store the result in a separate text file separated by space. For example, <a href="/kid/stay_healthy/">Staying Healthy</a> which has /kid/stay_healthy/ as...

2. UNIX for Dummies Questions & Answers

Extracting rows from a text file based on the first column

I have a tab delimited text file where the first column can take on three different values : 100, 150, 250. I want to extract all the rows where the first column is 100 and put them into a separate text file and so on. This is what my text file looks like now: 100 rs3794811 0.01 0.3434...

3. UNIX for Dummies Questions & Answers

Extracting rows from a text file based on the first column

I have a tab delimited text file where the first column can take on three different values : 100, 150, 250. I want to extract all the rows where the first column is 100 and put them into a separate text file and so on. This is what my text file looks like now: 100 rs3794811 0.01 0.3434 100...

4. UNIX for Dummies Questions & Answers

Extracting rows from a text file based on numerical values of a column

I have a text file where the second column is a list of numbers going from small to large. I want to extract the rows where the second column is smaller than or equal to 0.0001. My input: rs10082730 9e-08 12 46002702 rs2544081 1e-07 12 46015487 rs1425136 1e-06 7 35396742 rs2712590...

5. UNIX for Dummies Questions & Answers

Extracting rows from a space delimited text file based on the values of a column

I have a space delimited text file. I want to extract rows where the third column has 0 as a value and write those rows into a new space delimited text file. How do I go about doing that? Thanks!

6. UNIX for Dummies Questions & Answers

Extracting the last column of a text file

I would like to extract the last column of a text file but different rows of the text file have different numbers of columns. How do I go about doing that? Thanks!

7. Shell Programming and Scripting

Extracting the file name from the specified URL

Hello Everyone, I am trying to write a shell script(or Perl Script) that would do the following: I have a file that contains the following lines: File: https://ims-svnus.com/dev/DB/trunk/feeds/templates/shell_script.txt -r860...

8. UNIX for Dummies Questions & Answers

Extracting rows from a text file if the value of a column falls between a certain range

Hi, I have a file that looks like the following: 10 100080417 rs7915867 ILMN_1343295 12 6243093 7747537 10 100190264 rs2296431 ILMN_1343295 12 6643093 6647537 10 100719451 SNP94374 ILMN_1343295 12 6688093 7599537 ...

9. Shell Programming and Scripting

Extracting the column containing URL from a text file

I have the file like this: Timestamp URL Text 1331635241000 http://example.com Peoples footage at www.test.com,http://example4.com 1331635231000 http://example1.net crack the nuts http://example6.com 1331635280000 http://example2.net ...

10. Shell Programming and Scripting

Extracting the column containing URL from a text file

I have the file like this: Timestamp URL Text 1331635241000 http://example.com Peoples footage at www.test.com,http://example4.com 1331635231000 http://example1.net crack the nuts http://example6.com 1331635280000 http://example2.net ...

LEARN ABOUT CENTOS

locale::script

Locale::Script(3)					User Contributed Perl Documentation					 Locale::Script(3)

NAME

       Locale::Script - standard codes for script identification

SYNOPSIS

	  use Locale::Script;

	  $script  = code2script('phnx');		      # 'Phoenician'
	  $code    = script2code('Phoenician'); 	      # 'Phnx'
	  $code    = script2code('Phoenician',
				 LOCALE_CODE_NUMERIC);	      # 115

	  @codes   = all_script_codes();
	  @scripts = all_script_names();

DESCRIPTION

       The "Locale::Script" module provides access to standards codes used for identifying scripts, such as those defined in ISO 15924.

       Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 15924
       four-letter codes will be used.

SUPPORTED CODE SETS

       There are several different code sets you can use for identifying scripts. A code set may be specified using either a name, or a constant
       that is automatically exported by this module.

       For example, the two are equivalent:

	  $script = code2script('phnx','alpha');
	  $script = code2script('phnx',LOCALE_SCRIPT_ALPHA);

       The codesets currently supported are:

       alpha, LOCALE_SCRIPT_ALPHA
	   This is a set of four-letter (capitalized) codes from ISO 15924 such as 'Phnx' for Phoenician.  It also includes additions to this set
	   included in the IANA language registry.

	   The Zxxx, Zyyy, and Zzzz codes are not used.

	   This is the default code set.

       num, LOCALE_SCRIPT_NUMERIC
	   This is a set of three-digit numeric codes from ISO 15924 such as 115 for Phoenician.

ROUTINES

       code2script ( CODE [,CODESET] )
       script2code ( NAME [,CODESET] )
       script_code2code ( CODE ,CODESET ,CODESET2 )
       all_script_codes ( [CODESET] )
       all_script_names ( [CODESET] )
       Locale::Script::rename_script  ( CODE ,NEW_NAME [,CODESET] )
       Locale::Script::add_script  ( CODE ,NAME [,CODESET] )
       Locale::Script::delete_script  ( CODE [,CODESET] )
       Locale::Script::add_script_alias  ( NAME ,NEW_NAME )
       Locale::Script::delete_script_alias  ( NAME )
       Locale::Script::rename_script_code  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Script::add_script_code_alias  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Script::delete_script_code_alias  ( CODE [,CODESET] )
	   These routines are all documented in the Locale::Codes::API man page.

SEE ALSO

       Locale::Codes
	   The Locale-Codes distribution.

       Locale::Codes::API
	   The list of functions supported by this module.

       http://www.unicode.org/iso15924/
	   Home page for ISO 15924.

       http://www.iana.org/assignments/language-subtag-registry
	   The IANA language subtag registry.

AUTHOR

       See Locale::Codes for full author history.

       Currently maintained by Sullivan Beck (sbeck@cpan.org).

COPYRIGHT

	  Copyright (c) 1997-2001 Canon Research Centre Europe (CRE).
	  Copyright (c) 2001-2010 Neil Bowers
	  Copyright (c) 2010-2013 Sullivan Beck

       This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.16.3							    2013-06-03							 Locale::Script(3)