Shell Programming and Scripting

For example suppose I have a file which contains data as:
$cat data
800,2
100,9
700,3
100,9
200,8
100,3

Now I want the output as
200,8
700,3
800,2

Key is first three characters, I don't want any reords which are having duplicate keys.

Like sort +0.0 -0.3 data can we use similarly in uniq command?

Actual file contains more than 3 million records so I think any shell script will take lots of processing time. I want some fast command.

Please share your thoughts!

Thanks
Sumit

Code:

awk -F "," ' {
  cnt[$1] ++
  sav[$1] = $0
} 
END {
  for (x in sav)
     if (cnt[x] == 1)
       print sav[x]
}' your-file

if you have enough memory, this may works.
maybe the following is useful in memory tensive situation

Code:

awk -F "," '
NR == FNR {
  cnt[$1] ++
}
NR != FNR {
  if (cnt[$1] == 1)
    print $0
}' your-file your-file

Thanks a lot for your amazing code!

But it worked for sample data I have given.

Your first code is giving following error:
awk: 0602-590 Internal software error in the tostring function on

and second code really worked:
It took 3 min 16 seconds to process 3407871 records

Really cool! I was breaking my head in sort and uniq command !

Once again thank you!

Regards
Sumit

thank you it also worked!

But can you guys please explain the codes, so that I can understand what exactly it is doing? I really appreciate ur help!

Regards
Sumit

Only second code is working-- first and third one is giving error --
"awk: 0602-590 Internal software error in the tostring function on"

Thanks
sumit