I am doing KSH script to remove duplicate lines in a file. Let say the file has format below.
FileA
1253-6856
3101-4011
1827-1356
1822-1157
1822-1157
1000-1410
1000-1410
1822-1231
1822-1231
3101-4011
1822-1157
1822-1231
and I want to simply it with no duplicate line as file... (5 Replies)
Hi please help me how to remove duplicate lines in any file.
I have a file having huge number of lines.
i want to remove selected lines in it.
And also if there exists duplicate lines, I want to delete the rest & just keep one of them.
Please help me with any unix commands or even fortran... (7 Replies)
greetings,
i'm hoping there is a way to cat a file, remove duplicate lines and send that output to a new file. the file will always vary but be something similar to this:
please keep in mind that the above could be eight occurrences of each hostname or it might simply have another four of an... (2 Replies)
Hi,
I have two files with below data::
file1:-
123|aaa|ppp
445|fff|yyy
999|ttt|jjj
555|hhh|hhh
file2:-
445|fff|yyy
555|hhh|hhh
The records present in file1, not present in file 2 should be writtent to the out put file.
output:-
123|aaa|ppp
999|ttt|jjj
Is there any one line... (3 Replies)
Hey guys, need some help to fix this script. I am trying to remove all the duplicate lines in this file.
I wrote the following script, but does not work. What is the problem?
The output file should only contain five lines:
Later! (5 Replies)
hi,
Please help me to write a command to delete duplicate lines from a file. And the size of file is 50 MB. How to remove duplicate lins from such a big file. (6 Replies)
Hi,
I have a csv file which contains some millions of lines in it.
The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line).
I don't want to use any pattern from the Header as I have some... (7 Replies)
Dear community,
I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns
The data are like this:
Region 23/11/2014 09:11:36 41752
Medio 23/11/2014 03:11:38 4132
Info 23/11/2014 05:11:09 4323... (2 Replies)
Hi, all
I have a csv file that I would like to remove duplicate lines based on 1st field and sort them by the 1st field. If there are more than 1 line which is same on the 1st field, I want to keep the first line of them and remove the rest. I think I have to use uniq or something, but I still... (8 Replies)
Discussion started by: refrain
8 Replies
LEARN ABOUT OSX
bytes
bytes(3pm) Perl Programmers Reference Guide bytes(3pm)NAME
bytes - Perl pragma to force byte semantics rather than character semantics
NOTICE
This pragma reflects early attempts to incorporate Unicode into perl and has since been superseded. It breaks encapsulation (i.e. it
exposes the innards of how the perl executable currently happens to store a string), and use of this module for anything other than
debugging purposes is strongly discouraged. If you feel that the functions here within might be useful for your application, this possibly
indicates a mismatch between your mental model of Perl Unicode and the current reality. In that case, you may wish to read some of the perl
Unicode documentation: perluniintro, perlunitut, perlunifaq and perlunicode.
SYNOPSIS
use bytes;
... chr(...); # or bytes::chr
... index(...); # or bytes::index
... length(...); # or bytes::length
... ord(...); # or bytes::ord
... rindex(...); # or bytes::rindex
... substr(...); # or bytes::substr
no bytes;
DESCRIPTION
The "use bytes" pragma disables character semantics for the rest of the lexical scope in which it appears. "no bytes" can be used to
reverse the effect of "use bytes" within the current lexical scope.
Perl normally assumes character semantics in the presence of character data (i.e. data that has come from a source that has been marked as
being of a particular character encoding). When "use bytes" is in effect, the encoding is temporarily ignored, and each string is treated
as a series of bytes.
As an example, when Perl sees "$x = chr(400)", it encodes the character in UTF-8 and stores it in $x. Then it is marked as character data,
so, for instance, "length $x" returns 1. However, in the scope of the "bytes" pragma, $x is treated as a series of bytes - the bytes that
make up the UTF8 encoding - and "length $x" returns 2:
$x = chr(400);
print "Length is ", length $x, "
"; # "Length is 1"
printf "Contents are %vd
", $x; # "Contents are 400"
{
use bytes; # or "require bytes; bytes::length()"
print "Length is ", length $x, "
"; # "Length is 2"
printf "Contents are %vd
", $x; # "Contents are 198.144"
}
chr(), ord(), substr(), index() and rindex() behave similarly.
For more on the implications and differences between character semantics and byte semantics, see perluniintro and perlunicode.
LIMITATIONS
bytes::substr() does not work as an lvalue().
SEE ALSO
perluniintro, perlunicode, utf8
perl v5.16.2 2012-08-26 bytes(3pm)