Hi all
I have a file which looks like this

1234|1|Jon|some text|some text
1234|2|Jon|some text|some text
3453|5|Jon|some text|some text
6533|2|Kate|some text|some text
4567|3|Chris|some text|some text
4567|4|Maggie|some text|some text
8764|6|Maggie|some text|some text

So the output would look like this
1234|1|Jon|some text|some text
1234|2|Jon|some text|some text
3453|5|Jon|some text|some text
4567|4|Maggie|some text|some text
8764|6|Maggie|some text|some text

Try...

$ cat file1
1234|1|Jon|some text|some text
1234|2|Jon|some text|some text
3453|5|Jon|some text|some text
6533|2|Kate|some text|some text
4567|3|Chris|some text|some text
4567|4|Maggie|some text|some text
8764|6|Maggie|some text|some text

$ awk 'NR==FNR{a[$3]++;next}a[$3]>1' FS='|' file1 file1
1234|1|Jon|some text|some text
1234|2|Jon|some text|some text
3453|5|Jon|some text|some text
4567|4|Maggie|some text|some text
8764|6|Maggie|some text|some text

$

fortunately it doesn't do any anything on my file
so even putting into a file it returns an empty file

oooppsssss ... my mistake
I only put the file1 once
why we need to put it twice? is it for comparison?

Thanks for your help

---------- Post updated at 08:31 AM ---------- Previous update was at 08:16 AM ----------

i tested it on my documents

somehow, it does not delete all the single lines so I do steel have unique data
on the other hand it deletes one row from the non-unique ones as well so if i have two james on file one, in output i have 1 james only

any suggestion?

Hi,

Try this one,

awk 'BEGIN{FS="|";}{a[$3]++;if(a[$3]==2)print v[$3] ORS $0;if(a[$3]>2)print;v[$3]=$0;}' file

Dear Ranga

its worked as a charm

Thank you so much
Cheers
A-V

Hi Ranga,

Good One but can you please compeltly how this logic works?

awk 'BEGIN{FS="|";}{a[$3]++;if(a[$3]==2)print v[$3] ORS $0;if(a[$3]>2)print;v[$3]=$0;}' file

a[$3]++; - Store the no of repeat counts with name as a key.
v[$3]=$0; - store the previous line
if(a[$3]==2) - If repeat count is more than one(must be 2), then print 
previous line(first occurence) and current line(second occurence).
if(a[$3]>2) - Just print the current line.