UNIX for Dummies Questions & Answers

Background:
I have a file of thousands of potential SSR primers from Batch Primer 3.
I can't use primers that will contain the same sequence ID or sequence as another primer.
I have some basic shell scripting skills, but not enough to handle this.

What you need to know:
I need to remove the entire line(row) if its entry in column 3 or 13 is not unique when compared to the rest of its column. Or, I need to cat all lines that have a unique entry in columns 3 and 13 to a new file.

Note: I can't just remove the duplicate value, I have to remove the whole row after checking a value in that row against the rest of its column.

Example data is attached. Red values are duplicates.

Thank you very very very much!

Try something like:

Code:

awk -F, 'NR==FNR{A[$3]++; B[$13]++; next} A[$3]==1 && B[$13]==1' infile infile

Note: the input file is specified twice. I have used a comma separator here as field separator, so for that to work you need to use a comma as field separator when you export the spreadsheet.

It is untested since there is no text sample... Please post one.