Continued trouble matching fields in different files and selective field printing ([g]awk)

01-06-2018

Registered User

58, 2

Join Date: Aug 2014

Last Activity: 6 April 2020, 3:03 PM EDT

Posts: 58

Thanks Given: 61

Thanked 2 Times in 2 Posts

Quote:

Originally Posted by Don Cragun

If you are using only $1 & $2 as a key into file1 to match against $2 & $5 as a key into file2 and there those keys do not uniquely identify which record is to be processed when the data associated with those keys are not unique, there isn't anything I can do to help you. When there are multiple records with the same keys, you need to clearly describe how your code is supposed to determine which of those records are supposed to be paired.

The way the code is written now, the last record with a given key in file1 will be matched with all records with that key in file2. We now know that that isn't what you want. How do you uniquely describe a key that can be used to get what you do want? Or, in other words, what do you want to add to your current keys so your keys will uniquely identify the records you want to match in both files?

Sorry Don, as I am so new to this I did not realize that a repeat key item would cause this behavior until after I had tested the code you helped me modify. I thought (erroneously) that the order in which they appear (as Scrutinizer alludes to below) would be enough to disambiguate the two items in the array. As someone not from a math/science background (I'm in the Humanities) I do not fully understand the behavior of associative arrays yet (I understand the concept, but how to bring that to bear in coding has been a challenge for me). Nevertheless, I really appreciate your help.

---------- Post updated at 09:05 AM ---------- Previous update was at 09:04 AM ----------

Thank you Scrutinizer so much for this. I will have to test it further in a bit, but an initial run looks very promising. Ironically, I began this thread attempting to introduce a unique field in File1 as an additional, disambiguating search criteria for a part of another text processing need I had that was running aground due to repeat items in a field. In that other process I attempted multiple incrementing and count functions to no avail. I believe that your help in the proper use of the increment op within this code will help me solve that other issue as well. I appreciate your help very much.

Quote:

Originally Posted by Scrutinizer

The fact that combinations of $2 and $5 can occur multiple times is a new condition that was not apparent in the first post. So what is the criterion for what goes with what? Is it the order in which they appear in both files or does $3 in file2 play a role somehow?

Assuming it is the former of these two possibillties, try this modification:

Code:

awk 'NR==FNR {A[$1,$2,C1[$1,$2]++]=$0; next} ($2,$5,0) in A{print $1,A[$2,$5,C2[$2,$5]++]}' file1 file2

which (with the new sample in post #5) produces:

Code:

120 PS012,007 DKJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13  13   2      -1      -1      -1    0  521     0
122 PS012,007 S>M                 0   2 -1 -1 -1  1 -1   -1 -1  0  2     3   2   0  -1      -1      -1      -1   -1   -1    -1
123 PS012,007 GBJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13   2  -1      -1      -1      -1    0  502     0
125 PS012,007 D                  -1   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  519     0
126 PS012,007 BXR                 0   1  0 17  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0
127 PS012,007 B                   0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
128 PS012,007 >R<                 0   2 -1 -1 -1  1 -1   -1 -1  0  1     3   2   5   2      -1      -1      -1    0  505     0
133 PS012,007 W                   0   6 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
134 PS012,007 DKJ                 0   1  6 18  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0

==edit:==
The above code can create problems if it is possible combinations only appear once in file1 and twice in file2, which would lead to half lines being printed.
In that case it could be modified to this:

Code:

awk 'NR==FNR {A[$1,$2,++C1[$1,$2]]=$0; next} ($2,$5,++C2[$2,$5]) in A{print $1,A[$2,$5,C2[$2,$5]]}' file1 file2

So that would mean that a second occurrence in file2 would not get printed if it only appears once in file 1

jvoot

View Public Profile for jvoot

Find all posts by jvoot

UNIX for Beginners Questions & Answers

Continued trouble matching fields in different files and selective field printing ([g]awk)

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk for matching fields between files with repeated records

Discussion started by: jvoot

2. Shell Programming and Scripting

awk Selective printing of fields

Discussion started by: shunya

3. UNIX for Beginners Questions & Answers

Awk: matching multiple fields between 2 files

Discussion started by: beca123456

4. Shell Programming and Scripting

awk to combine all matching fields in input but only print line with largest value in specific field

Discussion started by: cmccabe

5. Shell Programming and Scripting

Printing entire field, if at least one row is matching by AWK

Discussion started by: Chulamakuri

6. Shell Programming and Scripting

Awk selective printing

Discussion started by: saint2006

7. Shell Programming and Scripting

AWK : Add Fields of lines with matching field

Discussion started by: DerSeb

8. Shell Programming and Scripting

AWK Matching Fields and Combining Files

Discussion started by: Michelangelo

9. Shell Programming and Scripting

selective printing awk

Discussion started by: saint2006

10. Shell Programming and Scripting

AWK - printing certain fields when field order changes in data file

Discussion started by: eric4