I am doing KSH script to remove duplicate lines in a file. Let say the file has format below.
FileA
1253-6856
3101-4011
1827-1356
1822-1157
1822-1157
1000-1410
1000-1410
1822-1231
1822-1231
3101-4011
1822-1157
1822-1231
and I want to simply it with no duplicate line as file... (5 Replies)
I have a log file "logreport" that contains several lines as seen below:
04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping
06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping
07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but... (18 Replies)
Hi please help me how to remove duplicate lines in any file.
I have a file having huge number of lines.
i want to remove selected lines in it.
And also if there exists duplicate lines, I want to delete the rest & just keep one of them.
Please help me with any unix commands or even fortran... (7 Replies)
greetings,
i'm hoping there is a way to cat a file, remove duplicate lines and send that output to a new file. the file will always vary but be something similar to this:
please keep in mind that the above could be eight occurrences of each hostname or it might simply have another four of an... (2 Replies)
Hi,
I have two files with below data::
file1:-
123|aaa|ppp
445|fff|yyy
999|ttt|jjj
555|hhh|hhh
file2:-
445|fff|yyy
555|hhh|hhh
The records present in file1, not present in file 2 should be writtent to the out put file.
output:-
123|aaa|ppp
999|ttt|jjj
Is there any one line... (3 Replies)
Hey guys, need some help to fix this script. I am trying to remove all the duplicate lines in this file.
I wrote the following script, but does not work. What is the problem?
The output file should only contain five lines:
Later! (5 Replies)
hi,
Please help me to write a command to delete duplicate lines from a file. And the size of file is 50 MB. How to remove duplicate lins from such a big file. (6 Replies)
Dear community,
I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns
The data are like this:
Region 23/11/2014 09:11:36 41752
Medio 23/11/2014 03:11:38 4132
Info 23/11/2014 05:11:09 4323... (2 Replies)
Hi, all
I have a csv file that I would like to remove duplicate lines based on 1st field and sort them by the 1st field. If there are more than 1 line which is same on the 1st field, I want to keep the first line of them and remove the rest. I think I have to use uniq or something, but I still... (8 Replies)
Discussion started by: refrain
8 Replies
LEARN ABOUT DEBIAN
text::csv::encoded::coder::encodeguess
Text::CSV::Encoded::Coder::EncodeGuess(3pm) User Contributed Perl Documentation Text::CSV::Encoded::Coder::EncodeGuess(3pm)NAME
Text::CSV::Encoded::Coder::EncodeGuess - Text::CSV::Encoded coder class using Encode::Guess
SYNOPSIS
use Text::CSV::Encoded coder_class => 'Text::CSV::Encoded::Coder::EncodeGuess';
use Spreadsheet::ParseExcel;
my $csv = Text::CSV::Encoded->new();
$csv->encoding( ['ucs2', 'ascii'] ); # guessing ucs2 or ascii?
$csv->encoding_to_combine('shiftjis');
my $excel = Spreadsheet::ParseExcel::Workbook->Parse( $file );
my $sheet = $excel->{Worksheet}->[0];
for my $row ( $sheet->{MinRow} .. $sheet->{MaxRow} ) {
my @fields;
for my $col ( $sheet->{MinCol} .. $sheet->{MaxCol} ) {
my $cell = $sheet->{Cells}[$row][$col];
push @fields, $cell->{Val};
}
$csv->combine( @fields ) or die;
print $csv->string, "
";
}
DESCRIPTION
This module is inherited from Text::CSV::Encoded::Coder::Encode.
USE
Except for 2 attributes, same as Text::CSV::Encoded::Coder::Encode.
encoding_in
$csv = $csv->encoding_in( $encoding_list_ref );
The accessor to an encoding for pre-parsing CSV strings. If no encoding is given, returns current $encoding, otherwise the object itself.
$encoding_list_ref = $csv->encoding_in()
When you pass a list reference, it might guess the encoding from the given list.
$csv->encoding_in( ['shiftjis', 'euc-jp', 'iso-20022-jp'] );
If it cannot guess the encoding, the first encoding of the list is used.
encoding
$csv = $csv->encoding( $encoding_list_ref );
$encoding_list_ref = $csv->encoding();
You can pass a list reference to this attribute only:
* For list data consumed by combine().
* For list reference returned by getline().
In other word, in "combine" and "print", it might guess an encoding for the passing list data. If it cannot guess the encoding, the first
encoding of the list is used.
SEE ALSO
Encode, Encode::Guess
AUTHOR
Makamaka Hannyaharamitu, <makamaka[at]cpan.org>
COPYRIGHT AND LICENSE
Copyright 2008-2010 by Makamaka Hannyaharamitu
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
perl v5.14.2 2010-04-26 Text::CSV::Encoded::Coder::EncodeGuess(3pm)