Extracting the column containing URL from a text file

07-16-2014

Registered User

17, 0

Join Date: Apr 2012

Last Activity: 1 August 2014, 5:20 PM EDT

Posts: 17

Thanks Given: 6

Thanked 0 Times in 0 Posts

Extracting the column containing URL from a text file

I have the file like this:

Timestamp URL Text 1331635241000 http://example.com Peoples footage at www.test.com,http://example4.com 1331635231000 http://example1.net crack the nuts http://example6.com 1331635280000 http://example2.net Loving thisEach column is tab separated. I need to extract only the URLs from column 2 and column 3 if in case of the no URLs then leave it empty for example to get the result like this:

URL Text http://example.com www.test.com,http://example4.com http://example1.net http://example6.com http://example2.net

I tried this script

Code:

awk 'BEGIN {FS="\t"} {print $2,$3}' file | grep -oP '(((http|https|ftp|gopher)|mailto)[.:][^ >"\t]*|www\.[-a-z0-9.]+)[^ .,;\t>">\):]'

This script can give me the all URLS in a single column without the header. Any suggestion to resolve this.

csim_mohan

View Public Profile for csim_mohan

Find all posts by csim_mohan

pactester(1) pactester(1) NAME
pactester - Tool to test proxy auto-config (pac) files. SYNOPSIS
pactester <-p pacfile> <-u url> [-h host] [-c client_ip] [-e] pactester <-p pacfile> <-f urlslist> [-c client_ip] [-e] DESCRIPTION
pactester is a tool to test proxy auto-config (pac) files. It returns the proxy config string for the given URL and the pac file. pactester uses pacparser C library for most of its functionality. OPTIONS
-p pacfile PAC file to test. Specify "-" to read from the standard input. -u url URL to test the PAC file for. -h host Host part of the URL. If not specified, it's determined from the URL. -c client_ip Client's IP address (as returned by the function myIpAddress() in PAC files). If not specified, it defaults to the IP address of the machine on which this tool is running. -e Enable Microsoft PAC extensions (dnsResolveEx, myIpAddressEx, isResolvableEx). -f urlslist A file containing the list of URLs to be tested. This is good for testing a PAC file against a set of URLs. EXAMPLES
To find out the proxy config string for the pac file "wpad.dat" and the URL "http://www.google.com": $ pactester -p wpad.dat -u http://www.google.com For a client with IP address 10.0.12.123: $ pactester -p wpad.dat -c 10.0.12.123 -u http://www.google.com For a pac file hosted at http://wpad/wpad.dat: $ curl -s http://wpad/wpad.dat | pactester -p - -u http://google.com BUGS
If you have come across a bug in pactester, please submit a bug report at http://code.google.com/p/pacparser/issues/list. AUTHOR
Written by Manu Garg (http://www.manugarg.com). RESOURCES
Homepage: http://code.google.com/p/pacparser. pactester(1)

Shell Programming and Scripting