UNIX for Dummies Questions & Answers

Is it possible to search through a large file full of rows and columns of text and retrieve only the rows that contain duplicates fields,

searchiing for duplicates on col4 & col6

Sample below

Col1 col2 col3 col4 col5 col6
G405H SURG FERGUSON SG00308258 01/16/52 GGHB
G405H ORTHO FERGUSON SG00308258 05/21/23 A&C
G405H ENT HOUGHTON SG03102407 04/22/70 GGHB
G405H ENT HOUGHTON SG00308258 10/08/60 GGHB
G405H GYN TAGGART SG03132070 05/15/53 GGHB

I would expect it the output to be

G405H SURG FERGUSON SG00308258 01/16/52 GGHB
G405H ENT HOUGHTON SG00308258 10/08/60 GGHB

input file = filename

Code:

G405H SURG FERGUSON SG00308258 01/16/52 GGHB
G405H ORTHO FERGUSON SG00308258 05/21/23 A&C
G405H ENT HOUGHTON SG03102407 04/22/70 GGHB
G405H ENT HOUGHTON SG00308258 10/08/60 GGHB
G405H GYN TAGGART SG03132070 05/15/53 GGHB

output

Code:

 
G405H ENT HOUGHTON SG00308258 10/08/60 GGHB
G405H SURG FERGUSON SG00308258 01/16/52 GGHB

Code:

 
sort  -k 4.1,4.10 -k 6.1,6.4 filename |
awk ' {
    if (arr[ $4 $6 ])
    {print arr[ $4 $6 ];print $0}    
    else { arr [$4 $6 ]	= $0 }
    }' filename | sort -u

Jim,

Once again ...a big thanks to you, Unfortunately for me though I use Data General - Unix and the commands don't seem to have the -k option, but I'll play around with it once I have figured out what parts of your code is doing what...the other thing is the real input file has many date columns and commas etc ...

Which makes it a little more complicated

cheers