Search Results

This requires a lot of memory depending on what you have in reference.txt
Simple awk which can be rewritten as something difficult to read for non-awkers.
We have posters who do that, which is okay...

In fact, what I applied is your logic from post#1, deploying the fact that the two expressions are mutually exclusive except for the $5 == $6 case. The result of a boolean expression is either 1...

I'm not yet sure I entirely grasp it, but that "grinding itself down to 0 lines" confirms my gut feeling about the processing. Unfortunately the few sample limes don't allow for a thorough testing....

e.g

If $3 > $4, print $2 (ID2) > exclusionlist.txt # This one will create or if exist,overwrite
If $3 < $4, print $1 (ID1) >> exclusionlist.txt # Then here you append...
If $3==$4 && $5 <...

Using awk:-
awk 'NR==FNR{A[$1];next}$1 in A' reference.txt data.txt
Using grep (no need to use cat):-
grep -f reference.txt data.txt

awk '{if(p[$2]<$3){p[$2]=$3;c[$2]=$1;}}END{for(i in c){print c[i]}}' example.data

Output:

113
112
118
115

sort -n -k2 -k3 infile | awk '!a[$2]++ && l {print l} {l=$1}; END {print l}'

Hello aberg,

If you are not worried about the order of the printing of 1st field then following may help you in same.
Please let me know how it goes then.

awk...

Can it be that your input files are in DOS format (contain carriage returns)?

Try:
tr -d '\r' file > file.new

first and on both files and try again...

---
Otherwise there may be excess...

awk -F: 'NR==FNR{a[$1]=$2;next}$1 in a{$1=a[$1]}1' ref_file file1