Hi,
I am processing a file and would like to delete duplicate records as indicated by one of its column. e.g.
COL1 COL2 COL3
A 1234 1234
B 3k32 2322
C Xk32 TTT
A NEW XX22
B 3k32 ... (7 Replies)
hii i have a huge amt of data stored in a file.Here in this file i need to remove duplicates rows in such a way that the last column has different data & i must check for greatest among last colmn data & print the largest data along with other entries but just one of other duplicate entries is... (16 Replies)
My input file:
AVI.out <detail>named as the RRM .</detail>
AVI.out <detail>Contains 1 RRM .</detail>
AR0.out <detail>named as the tellurite-resistance.</detail>
AWG.out <detail>Contains 2 HTH .</detail>
ADV.out <detail>named as the DENR family.</detail>
ADV.out ... (10 Replies)
I am a newbie to shell scripting ..
I have a .csv file. It has 1000 some rows and about 7 columns...
but before I insert this data to a table I have to parse it and clean it ..basing on the value of the first column..which a string of phone number type...
example below..
column 1 ... (2 Replies)
Hello,
I am new to shell scripting. I have a huge file with multiple columns for example:
I have 5 columns below.
HWUSI-EAS000_29:1:105 + chr5 76654650 AATTGGAA HHHHG
HWUSI-EAS000_29:1:106 + chr5 76654650 AATTGGAA B@HYL
HWUSI-EAS000_29:1:108 + ... (4 Replies)
Hi,
I want to remove duplicate records including the first line based on column1. For example
inputfile(filer.txt):
-------------
1,3000,5000
1,4000,6000
2,4000,600
2,5000,700
3,60000,4000
4,7000,7777
5,999,8888
expected output:
----------------
3,60000,4000
4,7000,7777... (5 Replies)
I was reading this thread. It looks like a simpler way to say this is to only keep uniq lines based on field or column 1.
https://www.unix.com/shell-programming-scripting/165717-removing-duplicate-records-file-based-single-column.html
Can someone explain this command please? How are there no... (5 Replies)
I have 2 files,
file01= 7 columns, row unknown (but few)
file02= 7 columns, row unknown (but many)
now I want to create an output with the first field that is shared in both of them and then subtract the results from the rest of the fields and print there
e.g.
file 01
James|0|50|25|10|50|30... (1 Reply)
Dear fellows, I need your help.
I'm trying to write a script to convert a single column into multiple rows.
But it need to recognize the beginning of the string and set it to its specific Column number.
Each Line (loop) begins with digit (RANGE).
At this moment it's kind of working, but it... (6 Replies)
Dear members, I need to filter a file based on the 8th column (that is id), and does not mather the other columns, because I want just one id (1 line of each id) and remove the duplicates lines based on this id (8th column), and does not matter wich duplicate will be removed.
example of my file... (3 Replies)
Discussion started by: clarissab
3 Replies
LEARN ABOUT DEBIAN
dbix::class::helper::schema::lintcontents
DBIx::Class::Helper::Schema::LintContents(3pm) User Contributed Perl Documentation DBIx::Class::Helper::Schema::LintContents(3pm)NAME
DBIx::Class::Helper::Schema::LintContents - Check the data in your database match your constraints
VERSION
version 2.013002
SYNOPSIS
package MyApp::Schema;
use parent 'DBIx::Class::Schema';
__PACKAGE__->load_components('Helper::Schema::LintContents');
1;
And later, somewhere else:
say "Incorrectly Null Users:";
for ($schema->null_check_source_auto('User')->all) {
say '* ' . $_->id
}
say "Duplicate Users:";
my $duplicates = $schema->dup_check_source_auto('User');
for (keys %$duplicates) {
say "Constraint: $_";
for ($duplicates->{$_}->all) {
say '* ' . $_->id
}
}
say "Users with invalid FK's:";
my $invalid_fks = $schema->fk_check_source_auto('User');
for (keys %$invalid_fks) {
say "Rel: $_";
for ($invalid_fks->{$_}->all) {
say '* ' . $_->id
}
}
DESCRIPTION
Some people think that constraints make their databases slower. As silly as that is, I have been in a similar situation! I'm here to help
you, dear developers! Basically this is a suite of methods that allow you to find violated "constraints." To be clear, the constraints I
mean are the ones you tell DBIx::Class about, real constraints are fairly sure to be followed.
METHODS
fk_check_source
my $busted = $schema->fk_check_source(
'User',
'Group',
{ group_id => 'id' },
);
"fk_check_source" takes three arguments, the first is the from source moniker of a relationship. The second is the to source or source
moniker of a relationship. The final argument is a hash reference representing the columns of the relationship. The return value is a
resultset of the from source that do not have a corresponding to row. To be clear, the example given above would return a resultset of
"User" rows that have a "group_id" that points to a "Group" that does not exist.
fk_check_source_auto
my $broken = $schema->fk_check_source_auto('User');
"fk_check_source_auto" takes a single argument: the source to check. It will check all the foreign key (that is, "belongs_to")
relationships for missing... "foreign" rows. The return value will be a hashref where the keys are the relationship name and the values
are resultsets of the respective violated relationship.
dup_check_source
my $smashed = $schema->fk_check_source( 'Group', ['id'] );
"dup_check_source" takes two arguments, the first is the source moniker to be checked. The second is an arrayref of columns that "should
be" unique. The return value is a resultset of the source that duplicate the passed columns. So with the example above the resultset
would return all groups that are "duplicates" of other groups based on "id".
dup_check_source_auto
my $ruined = $schema->dup_check_source_auto('Group');
"dup_check_source_auto" takes a single argument, which is the name of the resultsource in which to check for duplicates. It will return a
hashref where they keys are the names of the unique constraints to be checked. The values will be resultsets of the respective duplicate
rows.
null_check_source
my $blarg = $schema->null_check_source('Group', ['id']);
"null_check_source" tales two arguments, the first is the name of the source to check. The second is an arrayref of columns that should
contain no nulls. The return value is simply a resultset of rows that contain nulls where they shouldn't be.
null_check_source_auto
my $wrecked = $schema->null_check_source_auto('Group');
"null_check_source_auto" takes a single argument, which is the name of the resultsource in which to check for nulls. The return value is
simply a resultset of rows that contain nulls where they shouldn't be. This method automatically uses the configured columns that have
"is_nullable" set to false.
AUTHOR
Arthur Axel "fREW" Schmidt <frioux+cpan@gmail.com>
COPYRIGHT AND LICENSE
This software is copyright (c) 2012 by Arthur Axel "fREW" Schmidt.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.
perl v5.14.2 2012-06-18 DBIx::Class::Helper::Schema::LintContents(3pm)