Shell Programming and Scripting

Hello,
I have a file Pairs.txt composed of 3 columns. For simplicity, this is the file:

A B C
B D W
X Y Z
Z W K
B A C
Y J Q
F M P
Y X Z

I am trying to reduce the file such that I only keep the first occurrence of any "pair". In the example above, I would only want to keep one occurrence of A-B, and I would want to delete B-A. The same idea for example X-Y above. (Note that I would still want to keep the last column, so if I am keeping A-B, then I still want the entire row of A-B, so I would keep A B C).

As such, the desired output:
A B C
B D W
X Y Z
Z W K
Y J Q
F M P

The only solution I could think of is to:
1. Read the entire file
2. Use a loop to read each row X, then reverse the first 2 columns of row X
3. Search for the occurrence/repeat of that reversal in the remainder of the file
3a. If it occurs, then delete this row X
3b. If it doesn't, keep row X

This can be done with C++ if I must, but I am sure there is a faster way of doing this in a script?
Is there an easier solution, and a way to implement this in a shell or awk script?

Thank you in advance!
DG

Code:

awk '{tmp1=$1 $2;
         tmp2=$2 $1;
         if(tmp1 in arr || tmp2 in arr) {next}
         arr[$1 $2]=$0
       }
       END {for(i in arr) { print arr[i]}}'  inputfile

Start with this..

Try the following AWK program :

Code:

awk '
! Pairs[$1, $2] {
   Pairs[$1, $2] = Pairs[$2, $1] = 1;
   print $0
}
' info.txt

Input file info.txt :

Code:

A B C
B D W
X Y Z
Z W K
B A C
Y J Q
F M P
Y X Z

Output :

Code:

A B C
B D W
X Y Z
Z W K
Y J Q
F M P

Jean-Pierre.

---------- Post updated at 21:44 ---------- Previous update was at 21:40 ----------

Just a little problem with this solution, the two following records are considered as pairs:

Code:

XY Z A
X YZ B

Jean-Pierre.

Thank you both for the speedy replies!
I tried both of your suggestions and they both work fine with me.

I am just slightly confused about Jean-Pierre's comment for the solution of jim mcnamara,
if we have the case of
XY Z A
X YZ B

I guess it can be accepted, because I am only comparing the first two columns.
In the example above, are you suggesting there is a problem if we are comparing 3 pairs of variables?
Technically both solutions work if we assume (XY) is one item, ie, XY does not represent X+Y. In this case we are still reverting back to comparing 2 columns. Right?

Thanks,
DG

The jim solution use an array to memorize the pairs formed by the 2 first columns.
The problem is that the key used is the concatenation of the two columns, so the 2 lines gives the same key that is XY+Z=X+YZ=XYZ.

Jean-Pierre.

perl may help you some.

Code:

while(<DATA>){
	my @tmp = sort split;
	my $key = join "", @tmp;
	if(not exists $hash{$key}){
		$hash{$key}=1;
		print;
	}
}
__DATA__
A B C
B D W
X Y Z
Z W K
B A C
Y J Q
F M P
Y X Z

Code:

perl -lane 'print unless $h{join " ", sort @F[0,1]}++' Pairs.txt