Extracting the column containing URL from a text file

07-16-2014

Registered User

17, 0

Join Date: Apr 2012

Last Activity: 1 August 2014, 5:20 PM EDT

Posts: 17

Thanks Given: 6

Thanked 0 Times in 0 Posts

Extracting the column containing URL from a text file

I have the file like this:

Timestamp URL Text 1331635241000 http://example.com Peoples footage at www.test.com,http://example4.com 1331635231000 http://example1.net crack the nuts http://example6.com 1331635280000 http://example2.net Loving thisEach column is tab separated. I need to extract only the URLs from column 2 and column 3 if in case of the no URLs then leave it empty for example to get the result like this:

URL Text http://example.com www.test.com,http://example4.com http://example1.net http://example6.com http://example2.net

I tried this script

Code:

awk 'BEGIN {FS="\t"} {print $2,$3}' file | grep -oP '(((http|https|ftp|gopher)|mailto)[.:][^ >"\t]*|www\.[-a-z0-9.]+)[^ .,;\t>">\):]'

This script can give me the all URLS in a single column without the header. Any suggestion to resolve this.

csim_mohan

View Public Profile for csim_mohan

Find all posts by csim_mohan

DPKG-WWW(1) General Commands Manual DPKG-WWW(1) NAME
dpkg-www - program to remotely open a WWW Debian package browser SYNOPSIS
dpkg-www [-s|--stdout] [-h|--host host] [query] DESCRIPTION
dpkg-www is used to remotely control a WEB browser and open a dpkg URL on the local host from an interactive shell or script. If the command is invoked while running under X-window the script will try to find an installed X browser to open the specified URL. If not running under X it will try to use a text browser instead. If Netscape is found and a Netscape instance is already running it will be asked to open the dpkg URL on localhost with the optional query supplied on the command line. If there is no browser running, it will start automatically a new one. The functionality provided by this program is identical to running a web browser with the -remote openURL(http://localhost/cgi-bin/dpkg) argument or opening the same URL from within the browser. dpkg-www-installer is an helper application which can configured in the WWW browser for web installation. It should never be invoked directly by the user. OPTIONS
-s, --stdout Redirect output to stdout. Requires one of the text browsers (lynx, lynx-ssl or links) installed. -h, --host host Send the query to a remote host, where dpkg-www must be installed. query Specifies an optional package name, an absolute pathname or a query argument which is passed to the dpkg cgi-bin. See dpkg-www(8) for more information about the use of the cgi. FILES
/etc/dpkg-www.conf ~/.dpkg-www Configuration files for dpkg-www. It is not necessary for these files to exist, there are sensible defaults for everything, but you can specify your preferred www browser with the DPKG_WWW_BROWSER variable, for example: DPKG_WWW_BROWSER=mozilla EXAMPLES
dpkg-www This would open a dpkg URL on localhost listing all the installed packages. dpkg-www bash This would open a dpkg URL asking info on the bash package. dpkg-www -h pisolo bash This would open a dpkg URL asking info on the bash package on host pisolo. dpkg-www 'dpkg*' This would open a dpkg URL listing all packages matching dpkg*. dpkg-www /bin/bash This would open a dpkg URL asking info on the package(s) owning the file /bin/bash . dpkg-www depends=svgalib This would open a dpkg URL listing all packages depending on svgalib. dpkg-www --stdout depends=awk | grep ^ii This would list on stdout all packages depending on awk and grep all lines of installed packages. SEE ALSO
dpkg(8), dpkg-www(8) AUTHOR
Massimo Dal Zotto <dz@debian.org>. Bugs should be reported via the normal Debian bug reporting system. LICENCE
dpkg-www is licensed under the GNU General Public License version 2. September 1, 2004 DPKG-WWW(1)

Shell Programming and Scripting