09-02-2011
Removing Duplicates from file
Hi Experts,
Please check the following new requirement. I got data like the following in a file.
HTML Code:
FILE_HEADER
01cbbfde7898410| 3477945| home| 1
01cbc275d2c122| 3478234| WORK| 1
01cbbe4362743da| 3496386| Rich Spare| 1
01cbc275d2c122| 3478234| WORK| 1
This is pipe separated file with column 2,3 as key columns. The file should be formatted to the following output files
1) All records other than the duplicates
HTML Code:
FILE_HEADER
01cbbfde7898410| 3477945| home| 1
01cbbe4362743da| 3496386| Rich Spare| 1
2) The dupicate key file
Any thoughts on this.
Note:- The 'FILE_HEADER' should be there in the first file.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
i have a file with some 1000 entries it will contain entries like
1000,ram
2000,pankaj
1001,rahim
1000,ram
2532,govind
2000,pankaj
3000,venkat
2532,govind
what i want is i want to extract only the distinct rows from this file
so my output should contain only
1000,ram... (2 Replies)
Discussion started by: trichyselva
2 Replies
2. Shell Programming and Scripting
I have data like this:
It's sorted by the 2nd field (TID).
envoy,90000000000000634600010001,04/11/2008,23:19:27,RB00266,0015,DETAIL,ERROR,
envoy,90000000000000634600010001,04/12/2008,04:23:45,RB00266,0015,DETAIL,ERROR,... (1 Reply)
Discussion started by: kinksville
1 Replies
3. UNIX for Dummies Questions & Answers
hey all,
I need some help.
I have a text file with names in it.
My target is that if a particular pattern exists in that file more than once..then i want to rename all the occurences of that pattern by alternate patterns..
for e.g if i have PATTERN occuring 5 times then i want to... (3 Replies)
Discussion started by: ashisharora
3 Replies
4. Shell Programming and Scripting
I have a log file with posts looking like this:
--
Messages can be delivered by different systems at different times. The id number is used to sort out duplicate messages. What I need is to strip the arrival time from each post, sort posts by id number, and reattach arrival time to respective... (2 Replies)
Discussion started by: Ilja
2 Replies
5. Shell Programming and Scripting
Hi,
I have a file that I want to change the format of. It is a large file in rows but I want it to be comma separated (comma then a space).
The current file looks like this:
HI, Joe, Bob, Jack, Jack
After I would want to remove any duplicates so it would look like this:
HI, Joe,... (2 Replies)
Discussion started by: kylle345
2 Replies
6. Shell Programming and Scripting
Hi all,
I am working with a huge amount of files in a Linux environment and I was trying to filter my data. Here's what my data looks like
Name............................Size
OLUSDN.gf.gif-1.JPEG.......5 kb
LKJFDA01.gf.gif-1.JPEG.....3 kb
LKJFDA01.gf.gif-2.JPEG.....1 kb... (7 Replies)
Discussion started by: Error404
7 Replies
7. UNIX for Dummies Questions & Answers
Hi All,
I am merging files coming from 2 different systems ,while doing that I am getting duplicates entries in the merged file
I,01,000131,764,2,4.00
I,01,000131,765,2,4.00
I,01,000131,772,2,4.00
I,01,000131,773,2,4.00
I,01,000168,762,2,2.00
I,01,000168,763,2,2.00... (5 Replies)
Discussion started by: Sri3001
5 Replies
8. UNIX for Dummies Questions & Answers
I have been using grep to output whole lines using a pattern file with identifiers (fileA):
fig|562.2322.peg.1
fig|562.2322.peg.3
fig|562.2322.peg.3
fig|562.2322.peg.3
fig|562.2322.peg.7
From fileB with corresponding identifiers in the second column:
NODE_0 fig|562.2322.peg.1 peg ... (2 Replies)
Discussion started by: Mauve
2 Replies
9. Shell Programming and Scripting
i hav two files like
i want to remove/delete all the duplicate lines in file2 which are viz unix,unix2,unix3 (2 Replies)
Discussion started by: sagar_1986
2 Replies
10. Shell Programming and Scripting
i hav two files like
i want to remove/delete all the duplicate lines in file2 which are viz unix,unix2,unix3.I have tried previous post also,but in that complete line must be similar.In this case i have to verify first column only regardless what is the content in succeeding columns. (3 Replies)
Discussion started by: sagar_1986
3 Replies
LEARN ABOUT DEBIAN
html::formfu::constraint::dependon
HTML::FormFu::Constraint::DependOn(3pm) User Contributed Perl Documentation HTML::FormFu::Constraint::DependOn(3pm)
NAME
HTML::FormFu::Constraint::DependOn - Multi-field Dependency Constraint
SYNOPSIS
type: DependOn
name: foo
others: bar
DESCRIPTION
If a value is submitted for the field this constraint is attached to, then a value must also be submitted for all fields named in "others"
in HTML::FormFu::Constraint::_others.
By default, if any of the named fields in "others" in HTML::FormFu::Constraint::_others are missing, an error will be attached to each
missing field. This behaviour can be changed by setting any of "attach_errors_to_base" in HTML::FormFu::Constraint::_others,
"attach_errors_to_others" in HTML::FormFu::Constraint::_others or "attach_errors_to" in HTML::FormFu::Constraint::_others.
This constraint doesn't honour the "not()" value.
SEE ALSO
Is a sub-class of, and inherits methods from HTML::FormFu::Constraint::_others, HTML::FormFu::Constraint
HTML::FormFu
AUTHOR
Carl Franks "cfranks@cpan.org"
LICENSE
This library is free software, you can redistribute it and/or modify it under the same terms as Perl itself.
perl v5.14.2 2011-05-10 HTML::FormFu::Constraint::DependOn(3pm)