Search Results

Ok
Then this is what the doctor ordered. The command from the post #3 will filter duplicate values for 2 field.
If you need to select by the unique value of the entire line then so
awk '!T[$0]++'...

Maybe these are file names x2014 x2016 x2017 x2018 ?
Five hundred thousand rows for an array in AWK is a drop in the bucket
awk '!T[$2]++' x201[4678] > ONE_FILE

Hi
So your files look like this exactly?
cat file
x2014: 1 rs3094315 0 752566 G A
1 rs3131972 0 752721 G A
...
x2016: 0 ...