05-21-2009
awk script to remove duplicate rows in line
i have the long file more than one ns and www and mx in the line like .
i need the first ns record and first www and first mx from line .
the records are seperated with tthe ; i am try ing in awk scripting not getiing the solution.
NS,ns2.fastpark.net,216.8.177.29,windsor,on,can,-83.017,42.3;NS,ns1.fastpark.net,206.130.11.197,toronto,on,can,-79.443,43.751;WWW,
www.0--00--0.com,216.8.179.24,windsor,on,can,-83.017,42.3
WW,
www.0--5.com,63.251.171.80,vancouver,wa,usa,-122.728,45.681;WWW,www.0--5.com,63.251.171.81,vancouver,wa,usa,-122.728,45.681;WWW,www.0--5.com,66.150.161.136,vancouver,wa,usa,-122.728,45.681;WWW,www.0--5.com,66.150.161.140,vancouver,wa,usa,-122.728,45.681;WWW,www.0--5.com,66.150.161.141,vancouver,wa,usa,-122.728,45.681;WWW,www.0--5.com,69.25.27.170,vancouver,wa,usa,-122.728,45.681;WWW,www.0--5.com,69.25.27.173,vancouver,wa,usa,-122.728,45.681;MX,m1.dnsix.com,63.251.171.169,vancouver,wa,usa,-122.728,45.681
I need out put like these
NS,ns2.fastpark.net,216.8.177.29,windsor,on,can,-83.017,42.3;
WWW,
www.0--00--0.com,216.8.179.24,windsor,on,can,-83.017,42.3
WW,
www.0--5.com,63.251.171.80,vancouver,wa,usa,-122.728,45.681;
MX,m1.dnsix.com,63.251.171.169,vancouver,wa,usa,-122.728,45.681
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi,
I have a scenario here where I have created a flatfile with the below mentioned information. File as you can see is dispalyed in three columns
1st column is FileNameString
2nd column is Report_Name (this has spaces)
3rd column is Flag
Result file needed is, removal of duplicate... (1 Reply)
Discussion started by: Student37
1 Replies
2. UNIX for Dummies Questions & Answers
Hi,
I am processing a file and would like to delete duplicate records as indicated by one of its column. e.g.
COL1 COL2 COL3
A 1234 1234
B 3k32 2322
C Xk32 TTT
A NEW XX22
B 3k32 ... (7 Replies)
Discussion started by: risk_sly
7 Replies
3. Shell Programming and Scripting
Hi,
I have a log file having size of 48mb.
For such a large log file. I want to get the message in a particular format which includes only unique error and exception messages.
The following things to be done :
1) To remove all the date and time from the log file
2) To remove all the... (1 Reply)
Discussion started by: Pank10
1 Replies
4. Ubuntu
Hi every body
I have some text file with a lots of duplicate rows like this:
165.179.568.197
154.893.836.174
242.473.396.153
165.179.568.197
165.179.568.197
165.179.568.197
154.893.836.174
how can I delete the repeated rows?
Thanks
Saeideh (2 Replies)
Discussion started by: sashtari
2 Replies
5. Shell Programming and Scripting
I have some data that looks like,
1 3300665.mol 3300665 5177008 102.093
2 3300665.mol 3300665 5177008 102.093
3 3294015.mol 3294015 5131552 102.114
4 3294015.mol 3294015 5131552 102.114
5 3293734.mol 3293734 5129625 104.152
6 3293734.mol ... (13 Replies)
Discussion started by: LMHmedchem
13 Replies
6. UNIX for Dummies Questions & Answers
Hello, I'm trying to delete duplicates when there are more than 10 duplicates, based on the value of the first column.
e.g.
a 1
a 2
a 3
b 1
c 1
gives
b 1
c 1
but requires 11 duplicates before it deletes.
Thanks for the help
Video tutorial on how to use code tags in The UNIX... (11 Replies)
Discussion started by: informaticist
11 Replies
7. Shell Programming and Scripting
Dear members, I need to filter a file based on the 8th column (that is id), and does not mather the other columns, because I want just one id (1 line of each id) and remove the duplicates lines based on this id (8th column), and does not matter wich duplicate will be removed.
example of my file... (3 Replies)
Discussion started by: clarissab
3 Replies
8. Shell Programming and Scripting
I want to duplicate each row in my file
Egfile.txt
Name State Age
Jack NJ 34
John MA 23
Jessica FL 45
I want the code to produce this output
Name State Age
Jack NJ 34
Jack NJ 34
John MA 23
John MA 23
Jessica FL 45
Jessica FL 45 (6 Replies)
Discussion started by: sidnow
6 Replies
9. UNIX for Dummies Questions & Answers
Hi all,
I've got a file that has 12 fields. I've merged 2 files and there will be some duplicates in the following:
FILE:
1. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, 100
2. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, (EMPTY)
3. CDC, 54321, TEST3,... (4 Replies)
Discussion started by: tugar
4 Replies
10. Shell Programming and Scripting
I create a CGI in bash/html.
My awk script looks like :
echo "<table>"
for fn in /var/www/cgi-bin/LPAR_MAP/*;
do
echo "<td>"
echo "<PRE>"
awk -F',|;' -v test="$test" '
NR==1 {
split(FILENAME ,a,"");
}
$0 ~ test {
if(!header++){
... (12 Replies)
Discussion started by: Tim2424
12 Replies
LEARN ABOUT DEBIAN
www::mechanize::link
WWW::Mechanize::Link(3pm) User Contributed Perl Documentation WWW::Mechanize::Link(3pm)
NAME
WWW::Mechanize::Link - Link object for WWW::Mechanize
SYNOPSIS
Link object to encapsulate all the stuff that Mech needs but nobody wants to deal with as an array.
Constructor
new()
my $link = WWW::Mechanize::Link->new( {
url => $url,
text => $text,
name => $name,
tag => $tag,
base => $base,
attr => $attr_href,
} );
For compatibility, this older interface is also supported:
new( $url, $text, $name, $tag, $base, $attr_href )
Creates and returns a new "WWW::Mechanize::Link" object.
Accessors
$link->url()
URL from the link
$link->text()
Text of the link
$link->name()
NAME attribute from the source tag, if any.
$link->tag()
Tag name (one of: "a", "area", "frame", "iframe" or "meta").
$link->base()
Base URL to which the links are relative.
$link->attrs()
Returns hash ref of all the attributes and attribute values in the tag.
$link->URI()
Returns the URL as a URI::URL object.
$link->url_abs()
Returns a URI::URL object for the absolute form of the string.
SEE ALSO
WWW::Mechanize and WWW::Mechanize::Image
COPYRIGHT & LICENSE
Copyright 2004-2010 Andy Lester.
This program is free software; you can redistribute it and/or modify it under the terms of either:
o the GNU General Public License as published by the Free Software Foundation; either version 1, or (at your option) any later version,
or
o the Artistic License version 2.0.
perl v5.14.2 2011-08-05 WWW::Mechanize::Link(3pm)