Shell Programming and Scripting

Dear all
I have a large file w. ~ 10 million lines.
The first two cols have matching partners.
For example:
A A
A B
B B

or

A A
B A
B B

The matches may be separated by an unknown number of lines.

My intention is to group them and add a "group" value in the last col.

For example

A A A
A B A
B B A

or

A A A
B A A
B B A

Rest assured that only one of A B and B A will be present and not both.
Any help will be highly appreciated.
A may have matches in addition to B and any number of of them. But in all cases I would like to name the group with the first partner of the first instance, i.e. A in this case.
Any help will be highly appreciated.

How do you determine the group value? Why is the third line not B B B?

It would be helpful if you provided more examples from the file.

It might also help if you posted some real data in addition to the abbreviated, single-letter data.

Group value is determined by the first pair to be detected by the script.

If A A was the first pair, A is the first group value.
If A B was the first pair, A is the first group value.
If B A was the first pair, B is the first group value.
If B B was the first pair, B is the first group value.

I am sorting a large gene comparison data set, to us it hardly matters who the "group" is as far as the members are highly identical as the results indicate. This is only one of several analysis steps in my project.

Here is one set of instances of my data.

NC_002662.1|:1000271-1001206 NC_002662.1|:1000271-1001206 100.00 936 0 0 1 936 1 936 0.0 1814
NC_002662.1|:1000271-1001206 NC_008527.1|:1000752-1001687 88.60 947 86 21 1 936 1 936 0.0 957
NC_008527.1|:1000752-1001687 NC_008527.1|:1000752-1001687 100.00 936 0 0 1 936 1 936 0.0 1754

So it seems like the "group value" is always the same as the first field? If that's the case, why do you need to add another field?

No, if it was the first field all the time, I would never have posted this.
I kindly request you to look at my input again - if A B was encountered previously, when you next see B B it needs to be assigned to A.

I wanted to only provide a simple example but I guess I made it too simple and now appear not so smart.

Lets add some more.

Input

A A
A X
C D
E F
X L
A B
O O
P P
M N
B B

Output

A A A
A X A
C D C
E F E
X L X
A B A
O O O
T X X
E E E
P P P
M N M
B B A

My apologies, this is literally the first time I am posting questions in a programming forum. Please help me with further queries as you deem necessary.

Try this:

Code:

awk '
        $1 in group {
                print $0,group[$1]
                if ($2 in group) {
                        if (group[$1] != group[$2]) {
                                print $1" and "$2" are already in different groups!"
                        }
                } else {
                        group[$2]=group[$1]
                }
                next
        }
        $2 in group {
                print $0,group[$2]
                group[$1]=group[$2]
                next
        }
        {
                group[$1]=$1
                group[$2]=$1
                print $0,group[$1]
        }
' inputfile

I think you forgot to include the "T X" and "E E" lines in your example input data.

Note that the output is slightly different, e.g. T X A, not T X X because X is already in group A:

Code:

A A A
A X A
C D C
E F E
X L A
A B A
O O O
T X A
E E E
P P P
M N M
B B A

So what is the rule for determining the group?

When I "next see B B"? I haven't seen it before.

Why is that T X X and not T X T?

Why is that last line B B A and not B B B?

Does this do what you want?

Code:

awk '
{ group = (x[$1]) ? x[$1] : (x[$2]) ? x[$2] : $1 }
x[$1] || x[$2] { group = (x[$1]) ? x[$1] : x[$2] }
{print $0, group }
!x[$1] { x[$1] = group}
!x[$2] { x[$2] = group }
' "$FILE"

##