UNIX for Dummies Questions & Answers

Hi,

I have a file with fields like below:

A;XYZ;102345;222
B;XYZ;123243;333
C;ABC;234234;444
D;MNO;103345;222
E;DEF;124243;333


desired output:

C;ABC;234234;444
D;MNO;103345;222
E;DEF;124243;333


ie, if the 4rth field is a duplicate.. i need only those records where the 3rd field value is greater .. or infact, more specificcally.. need those where the 2nd and 3rd digits of the 3rd field are greater.. Can we do this with awk?


Pls help in finding a solution where i can quickly process a file of around 50000 records

Thanks a lot in advance

Hi,

Try this one,

Code:

awk 'BEGIN{FS=";";OFS=";";}{split(/\;/,a,c[$4]);if(a[3] < $3 ){c[$4]=$0;}}END{for( i in c ){print c[i];}}' file

Cheers,
Ranga

Hi, try:

Code:

sort -t \; -k3.2,3.3rn infile | awk -F\; '!A[$4]++'

Make sure there are not spaces in the input file.

Thanks.. I did a random check and it looks to be working perfectly.. Could you please tell me what the "!A[$4]++ does ?

Thanks again..

Hi, for every line it creates an (associative) array element with field 4 as the index (if it does not exist yet) without a value (or 0 is you will). The exclamation mark negates that value so the outcome is 1 (true). The value of 1 in awk means perform the default action which is {print $0} so the entire line gets printed.

Afterwards the ++ comes into action and 1 is added to the array value, which now becomes 1. So that next time a line with the same value in $4 is encountered the value returned by the array is 1 which is then negated to 0 by the exclamation mark, so nothing will get printed.

Thanks a lot Scrutinizer and Rangarasan