I have this sample text file..
E643E32D00AB58B49926B3C9628793E5,907 ,9999,5/1/2004,867 ,12/31/2006,ACT,1,0,1
CA589E9EC9CDBABA560EE6BF77AA4DBE,907 ,8741,7/1/2006,867 ,7/31/2007,ACT,1,0,1
5DBD6FF7877F5F38C62658DA5E460E64,907 ,5141,10/1/2003,867 ,9/30/2008,ACT,1,0,1
DB392456D01E0BDEE374C7BD62C9301F,907 ,4213,7/1/2009,867 ,12/31/9999,ACT,1,0,1
E1D08EF15E28E729D354B2484DDF5DFB,907 ,1014,6/15/2010,809 ,6/15/2010,DEL,500001,0,500001
86487F19E6275AFAC66279077B94FDE3,907 ,1542,6/1/2009,867 ,12/31/9999,ACT,1,0,1
E45B7371EEC0D1AB00E1750B5BC661F7,907 ,5211,1/1/2004,867 ,12/31/2006,ACT,1,0,1
FCBAFE572C5E4BA29B3F8030BD480A94,907 ,6531,1/1/2003,867 ,12/31/2005,ACT,1,0,1
2345AD5D2BFB29C821C1BC3DE8B746A7,907 ,2711,1/1/2004,827 ,1/31/2305,ACT,1,0,1
2345AD5D2BFB29C821C1BC3DE8B746A7,907 ,2711,1/1/2004,867 ,1/31/2005,ACT,1,0,1
F30641D0918E6BD2BA0B13903B3EA012,907 ,1541,5/1/2007,867 ,8/31/2007,ACT,1,0,1
F30641D0918E6BD2BA0B13903B3EA012,907 ,1541,5/1/2007,867 ,8/31/2007,ACT,1,0,1
The last two lines are exact duplicates and the penultimate two lines are duplicates only for my keys which are columns 1 and 2.
when i tried the code provided above modifying it like this
sort -k1,2 f1.txt |sort -mu -k1,2
It just removes the line corresponding to this key F30641D0918E6BD2BA0B13903B3EA012,907
but the lines corresponding to the key 2345AD5D2BFB29C821C1BC3DE8B746A7,907 are not removed.
I do not want to use awk, since i will not be able to reuse it..
The keys might not be fixed.. I will be passing it as a variable..
My reusable code might look like
pk=1,2
sort -k1$pk f1.txt|sort -mu -k$pk
Please help..
Last edited by gpsridhar; 10-05-2010 at 12:25 PM..
Reason: Additional information provided