Hi all,
I have a tab-delimited file and want to remove identical lines, i.e. all of line 1,2,4 because the columns are the same as the columns in other lines. Any input is appreciated.
abc gi4597 9997 cgcgtgcg $%^&*()()*
abc gi4597 9997 cgcgtgcg $%^&*()()*
ttt ... (1 Reply)
I have a csv file that I would like to remove duplicate lines based on field 1 and sort. I don't care about any of the other fields but I still wanna keep there data intact. I was thinking I could do something like this but I have no idea how to print the full line with this. Please show any method... (8 Replies)
Hi,
How can I remove duplicates from a file based on group on other column? for example:
Test1|Test2|Test3|Test4|Test5
Test1|Test6|Test7|Test8|Test5
Test1|Test9|Test10|Test11|Test12
Test1|Test13|Test14|Test15|Test16
Test17|Test18|Test19|Test20|Test21
Test17|Test22|Test23|Test24|Test5
... (2 Replies)
Hi All,
i have input file like below...
CA009156;20091003;M;AWBKCA72;123;;CANADIAN WESTERN BANK;EDMONTON;;2300, 10303, JASPER AVENUE;;T5J 3X6;;
CA009156;20091003;M;AWBKCA72;321;;CANADIAN WESTERN BANK;EDMONTON;;2300, 10303, JASPER AVENUE;;T5J 3X6;;
CA009156;20091003;M;AWBKCA72;231;;CANADIAN... (2 Replies)
Hi ,
Some time i got duplicated value in my files ,
bundle_identifier= B
Sometext=ABC
bundle_identifier= A
bundle_unit=500
Sometext123=ABCD
bundle_unit=400
i need to check if there is a duplicated values or not if yes , i need to check if the value is A or B when Bundle_Identified ,... (2 Replies)
I have a 13gb file. It has the following columns:
The 3rd column is basically correlation values. I want to delete those rows which are repeated between the columns:
A B 0.04
B C 0.56
B B 1
A A 1
C D 1
C C 1
Desired Output: (preferably in a .csv format
A,B,0.04
B,C,0.56
C,D,1... (3 Replies)
Dear community,
I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns
The data are like this:
Region 23/11/2014 09:11:36 41752
Medio 23/11/2014 03:11:38 4132
Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies
LEARN ABOUT REDHAT
pdl::char
Char(3) User Contributed Perl Documentation Char(3)NAME
PDL::Char -- PDL subclass which allows reading and writing of fixed-length character strings as byte PDLs
SYNOPSIS
use PDL;
use PDL::Char;
my $pchar = PDL::Char->new( [['abc', 'def', 'ghi'],['jkl', 'mno', 'pqr']] );
$pchar->setstr(1,0,'foo');
print $pchar; # 'string' bound to "", perl stringify function
# Prints:
# [
# ['abc' 'foo' 'ghi']
# ['jkl' 'mno' 'pqr']
# ]
print $pchar->atstr(2,0);
# Prints:
# ghi
DESCRIPTION
This subclass of PDL allows one to manipulate PDLs of 'byte' type as if they were made of fixed length strings, not just numbers.
This type of behavior is useful when you want to work with charactar grids. The indexing is done on a string level and not a character
level for the 'setstr' and 'atstr' commands.
This module is in particular useful for writing NetCDF files that include character data using the PDL::NetCDF module.
FUNCTIONS
new
Function to create a byte PDL from a string, list of strings, list of list of strings, etc.
# create a new PDL::Char from a perl array of strings
$strpdl = PDL::Char->new( ['abc', 'def', 'ghij'] );
# Convert a PDL of type 'byte' to a PDL::Char
$strpdl1 = PDL::Char->new (sequence (byte, 4, 5)+99);
$pdlchar3d = PDL::Char->new([['abc','def','ghi'],['jkl', 'mno', 'pqr']]);
string
Function to print a character PDL (created by 'char') in a pretty format.
$char = PDL::Char->new( [['abc', 'def', 'ghi'], ['jkl', 'mno', 'pqr']] );
print $char; # 'string' bound to "", perl stringify function
# Prints:
# [
# ['abc' 'def' 'ghi']
# ['jkl' 'mno' 'pqr']
# ]
# 'string' is overloaded to the "" operator, so:
# print $char;
# should have the same effect.
setstr
Function to set one string value in a character PDL. The input position is the position of the string, not a character in the string. The
first dimension is assumed to be the length of the string.
The input string will be null-padded if the string is shorter than the first dimension of the PDL. It will be truncated if it is longer.
$char = PDL::Char->new( [['abc', 'def', 'ghi'], ['jkl', 'mno', 'pqr']] );
$char->setstr(0,1, 'foobar');
print $char; # 'string' bound to "", perl stringify function
# Prints:
# [
# ['abc' 'def' 'ghi']
# ['foo' 'mno' 'pqr']
# ]
$char->setstr(2,1, 'f');
print $char; # 'string' bound to "", perl stringify function
# Prints:
# [
# ['abc' 'def' 'ghi']
# ['foo' 'mno' 'f'] -> note that this 'f' is stored "f "
# ]
atstr
Function to fetch one string value from a PDL::Char type PDL, given a position within the PDL. The input position of the string, not a
character in the string. The length of the input string is the implied first dimension.
$char = PDL::Char->new( [['abc', 'def', 'ghi'], ['jkl', 'mno', 'pqr']] );
print $char->atstr(0,1);
# Prints:
# jkl
perl v5.8.0 2001-05-27 Char(3)