Quote:
|
Originally Posted by amit_sapre
Try out this one...
sed '$!N; /^\(.*\)\n\1$/!P; D'
# The first line of duplicate ones is only kept and rest are deleted.
Hope this will work faster than sort command.
I haven't tried on large files. |
Havn't tried your
sed. But doesnt it assume that all the entries are already sorted and then it removes the duplicates.
and/or
If the file is unsorted, then duplicate entries based on first line are removed. since
sed makes just one-pass through the file.
Or did I get it wrong ?
vino
