The UNIX and Linux Forums  
Hello and Welcome from United States to the UNIX and Linux Forums! Thank You for Visiting and Joining Our Global Community.

Go Back   The UNIX and Linux Forums > Top Forums > UNIX for Advanced & Expert Users
Reload this Page How to remove duplicate lines of a record without changing the order
.
google unix.com

Forums Register Forum RulesLinksAlbums FAQ Members List Calendar Search Today's Posts Mark Forums Read



Thread: How to remove duplicate lines of a record without changing the order
View Single Post in the UNIX and Linux Forums - Click on the Thread or Permalink to View Entire Thread -->
  #6 (permalink)  
Old 08-03-2008
era era is offline Forum Advisor  
Herder of Useless Cats (On Sabbatical)
  
 

Join Date: Mar 2008
Location: /there/is/only/bin/sh
Posts: 3,652
If the requirement is to keep the last instead of the first occurrence, it's only marginally harder.

Code:
perl -ne '$n{$_} = $.; END { print sort { $n{$a} <=> $n{$b} } keys %n }'
If the last line lacks a newline, that will count as a unique line. It's not terribly hard to fix, but I didn't want to complicate the script.