Find duplicated values in two columns out of three
hi!
could u help in the following? I have the data (long list!) that looks like (three coumns white space separated):
And i know that values in the first column are unique, whereas in the second in the third there are duplicates. In other words two different "rs" may correspond to same values in the 2nd and 3rd columns. I need to find the duplicates in 2 and 3 columns and then remove whole line that will contain one unique rs and duplicated values in 2 and 3 coulumns.
Thank u in advance! kush
Last edited by Scrutinizer; 11-01-2012 at 10:13 AM..
Reason: code tags
hi all,
i have a file contain multicolumns, this file is sorted by col2 and col3.
i want to remove the duplicated columns if the col2 and col3 are the same in another line.
example
fileA
AA BB CC DD
CC XX CC DD
BB CC ZZ FF
DD FF HH HH
the output is
AA BB CC DD
BB CC ZZ FF... (6 Replies)
Hi Guys...
Please Could you help me with the following ?
aaaa bbbb cccc sdsd
aaaa bbbb cccc qwer
as you can see, the 2 lines are matched in three fields...
how can I delete this pupicate ? I mean to delete the second one if 3 fields were duplicated ?
Thanks (14 Replies)
I can not figure out this one, so I turn to unix.com for help, I have a file, in which there are some lines containing continuously duplicate columns, like the following
adb abc abc asd adfj
123 123 123 345
234 444 444 444 444 444 23
and the output I want is
adb abc asd adfj
123 345... (5 Replies)
Hi. I have a problem that i can't seem to resolve. I need to create a script that list all the files, that are found recursively, with the same name.
For example if a file exists in more than one directory with the same name it list all the files that he founds with all the info. Could someone... (5 Replies)
Hi everyone,
I have file1 and file2 comma separated both.
file1 is:
Header1,Header2,Header3,Header4,Header5,Header6,Header7,Header8,Header9,Header10
Code7,,,,,,,,,
Code5,,,,,,,,,
Code3,,,,,,,,,
Code9,,,,,,,,,
Code2,,,,,,,,,file2... (17 Replies)
I have a text file that has three columns. But at the end of the text file, there are trailing lines that have missing second and third columns:
4 0.04972604 KLHL28
4 0.0497332 CSTB
4 0.04979822 AIF1
4 0.04983331 DECR2
4 0.04990344 KATNB1
4
4
4
4
How can I remove the trailing... (3 Replies)
Hello
I have a file as below
chr1 start ref alt code1 code2
chr1 18884 C CAAAA 2 0
chr1 135419 TATACA T 2 0
chr1 332045 T TTG 0 2
chr1 453838 T TAC 2 0
chr1 567652 T TG 1 0
chr1 602541 ... (2 Replies)
Hi All,
I am new to shell scripting. I have a requirement as part of my job to find out null/empty values in column 2 and column 3 from a CSV file and exit the further execution of script by displaying a simple error message.
I have developed a script to do this by reading various articles... (7 Replies)
Hi,
I have the following output from an Oracle SQL statement and I want to remove duplicated column values.
I know it is possible using Oracle analytical/statistical functions but unfortunately I don't know how to use any of those.
So now, I've gone to PLAN B using awk/sed maybe or any... (5 Replies)
Please help me to get required output for both scenario 1 and scenario 2 and need separate code for both scenario 1 and scenario 2
Scenario 1
i need to do below changes only when column1 is CR and column3 has duplicates rows/values. This inputfile can contain 100 of this duplicated rows of... (1 Reply)
Discussion started by: as7951
1 Replies
LEARN ABOUT DEBIAN
dbix::class::helper::schema::lintcontents
DBIx::Class::Helper::Schema::LintContents(3pm) User Contributed Perl Documentation DBIx::Class::Helper::Schema::LintContents(3pm)NAME
DBIx::Class::Helper::Schema::LintContents - Check the data in your database match your constraints
VERSION
version 2.013002
SYNOPSIS
package MyApp::Schema;
use parent 'DBIx::Class::Schema';
__PACKAGE__->load_components('Helper::Schema::LintContents');
1;
And later, somewhere else:
say "Incorrectly Null Users:";
for ($schema->null_check_source_auto('User')->all) {
say '* ' . $_->id
}
say "Duplicate Users:";
my $duplicates = $schema->dup_check_source_auto('User');
for (keys %$duplicates) {
say "Constraint: $_";
for ($duplicates->{$_}->all) {
say '* ' . $_->id
}
}
say "Users with invalid FK's:";
my $invalid_fks = $schema->fk_check_source_auto('User');
for (keys %$invalid_fks) {
say "Rel: $_";
for ($invalid_fks->{$_}->all) {
say '* ' . $_->id
}
}
DESCRIPTION
Some people think that constraints make their databases slower. As silly as that is, I have been in a similar situation! I'm here to help
you, dear developers! Basically this is a suite of methods that allow you to find violated "constraints." To be clear, the constraints I
mean are the ones you tell DBIx::Class about, real constraints are fairly sure to be followed.
METHODS
fk_check_source
my $busted = $schema->fk_check_source(
'User',
'Group',
{ group_id => 'id' },
);
"fk_check_source" takes three arguments, the first is the from source moniker of a relationship. The second is the to source or source
moniker of a relationship. The final argument is a hash reference representing the columns of the relationship. The return value is a
resultset of the from source that do not have a corresponding to row. To be clear, the example given above would return a resultset of
"User" rows that have a "group_id" that points to a "Group" that does not exist.
fk_check_source_auto
my $broken = $schema->fk_check_source_auto('User');
"fk_check_source_auto" takes a single argument: the source to check. It will check all the foreign key (that is, "belongs_to")
relationships for missing... "foreign" rows. The return value will be a hashref where the keys are the relationship name and the values
are resultsets of the respective violated relationship.
dup_check_source
my $smashed = $schema->fk_check_source( 'Group', ['id'] );
"dup_check_source" takes two arguments, the first is the source moniker to be checked. The second is an arrayref of columns that "should
be" unique. The return value is a resultset of the source that duplicate the passed columns. So with the example above the resultset
would return all groups that are "duplicates" of other groups based on "id".
dup_check_source_auto
my $ruined = $schema->dup_check_source_auto('Group');
"dup_check_source_auto" takes a single argument, which is the name of the resultsource in which to check for duplicates. It will return a
hashref where they keys are the names of the unique constraints to be checked. The values will be resultsets of the respective duplicate
rows.
null_check_source
my $blarg = $schema->null_check_source('Group', ['id']);
"null_check_source" tales two arguments, the first is the name of the source to check. The second is an arrayref of columns that should
contain no nulls. The return value is simply a resultset of rows that contain nulls where they shouldn't be.
null_check_source_auto
my $wrecked = $schema->null_check_source_auto('Group');
"null_check_source_auto" takes a single argument, which is the name of the resultsource in which to check for nulls. The return value is
simply a resultset of rows that contain nulls where they shouldn't be. This method automatically uses the configured columns that have
"is_nullable" set to false.
AUTHOR
Arthur Axel "fREW" Schmidt <frioux+cpan@gmail.com>
COPYRIGHT AND LICENSE
This software is copyright (c) 2012 by Arthur Axel "fREW" Schmidt.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.
perl v5.14.2 2012-06-18 DBIx::Class::Helper::Schema::LintContents(3pm)