Continued trouble matching fields in different files and selective field printing ([g]awk) Post: 303010722

Sponsored Content

Top Forums UNIX for Beginners Questions & Answers Continued trouble matching fields in different files and selective field printing ([g]awk) Post 303010722 by Scrutinizer on Saturday 6th of January 2018 02:26:00 AM

01-06-2018

Moderator

The fact that combinations of $2 and $5 can occur multiple times is a new condition that was not apparent in the first post. So what is the criterion for what goes with what? Is it the order in which they appear in both files or does $3 in file2 play a role somehow?

Assuming it is the former of these two possibillties, try this modification:

Code:

awk 'NR==FNR {A[$1,$2,C1[$1,$2]++]=$0; next} ($2,$5,0) in A{print $1,A[$2,$5,C2[$2,$5]++]}' file1 file2

which (with the new sample in post #5) produces:

Code:

120 PS012,007 DKJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13  13   2      -1      -1      -1    0  521     0
122 PS012,007 S>M                 0   2 -1 -1 -1  1 -1   -1 -1  0  2     3   2   0  -1      -1      -1      -1   -1   -1    -1
123 PS012,007 GBJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13   2  -1      -1      -1      -1    0  502     0
125 PS012,007 D                  -1   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  519     0
126 PS012,007 BXR                 0   1  0 17  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0
127 PS012,007 B                   0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
128 PS012,007 >R<                 0   2 -1 -1 -1  1 -1   -1 -1  0  1     3   2   5   2      -1      -1      -1    0  505     0
133 PS012,007 W                   0   6 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
134 PS012,007 DKJ                 0   1  6 18  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0

==edit:==
The above code can create problems if it is possible combinations only appear once in file1 and twice in file2, which would lead to half lines being printed.
In that case it could be modified to this:

Code:

awk 'NR==FNR {A[$1,$2,++C1[$1,$2]]=$0; next} ($2,$5,++C2[$2,$5]) in A{print $1,A[$2,$5,C2[$2,$5]]}' file1 file2

So that would mean that a second occurrence in file2 would not get printed if it only appears once in file 1

Last edited by Scrutinizer; 01-06-2018 at 12:59 PM..

This User Gave Thanks to Scrutinizer For This Post:

Scrutinizer

View Public Profile for Scrutinizer

Find all posts by Scrutinizer

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

AWK - printing certain fields when field order changes in data file

I'm hoping someone can help me on this. I have a data file that greatly simplified might look like this: sec;src;dst;proto 421;10.10.10.1;10.10.10.2;tcp 426;10.10.10.3;10.10.10.4;udp 442;10.10.10.5;10.10.10.6;tcp sec;src;fac;dst;proto 521;10.10.10.1;ab;10.10.10.2;tcp...

2. Shell Programming and Scripting

selective printing awk

Hi there my file looks like this 1 a b c d e f 2 a b b c d e f f g h e t t 3 a c b d e f 4 a b c i want to print the line which has the fields containing ONLY a b c, in this case the line 4. How can i awk it !!!? Many Thanks in advance!

3. Shell Programming and Scripting

AWK Matching Fields and Combining Files

Hello! I am writing a program to run through two large lists of data (~300,000 rows), find where rows in one file match another, and combine them based on matching fields. Due to the large file sizes, I'm guessing AWK will be the most efficient way to do this. Overall, the input and output I'm...

4. Shell Programming and Scripting

AWK : Add Fields of lines with matching field

Dear All, I would like to add values of a field, if the lines match in a certain field. Then I would like to divide the sum though the number of lines that have a matched field. This is the Input: Input: Test1 5 Test1 10 Test2 2 Test2 5 Test2 13 Test3 4 Output: Test1 7.5 Test1 7.5...

5. Shell Programming and Scripting

Awk selective printing

Hi, i need help to print number from different field INPUT: Student1 10 20 Student2 30 40 Student3 50 60 Student4 70 80 Desired Output: 1 20-30 2 40-50 3 60-70 Thank you!

6. Shell Programming and Scripting

Printing entire field, if at least one row is matching by AWK

Dear all, I have been trying to print an entire field, if the first line of the field is matching. For example, my input looks something like this. aaa ddd zzz 123 987 126 24 0.650 985 354 9864 0.32 0.333 4324 000 I am looking for a pattern,...

7. Shell Programming and Scripting

awk to combine all matching fields in input but only print line with largest value in specific field

In the below I am trying to use awk to match all the $13 values in input, which is tab-delimited, that are in $1 of gene which is just a single column of text. However only the line with the greatest $9 value in input needs to be printed. So in the example below all the MECP2 and LTBP1...

8. UNIX for Beginners Questions & Answers

Awk: matching multiple fields between 2 files

Hi, I have 2 tab-delimited input files as follows. file1.tab: green A apple red B apple file2.tab: apple - A;Z Objective: Return $1 of file1 if, . $1 of file2 matches $3 of file1 and, . any single element (separated by ";") in $3 of file2 is present in $2 of file1 In order to...

9. Shell Programming and Scripting

awk Selective printing of fields

Hi Gurus, I have following input file. I tried multiple awk combinations to print selected columns without success. HEX ID Name ver FLRGT Start Time Total Shared End Date ----- -------- --- ------ ------------------------ -------------- -------...

10. UNIX for Beginners Questions & Answers

awk for matching fields between files with repeated records

Hello all, I am having trouble with what should be an easy task, but seem to be missing something fundamental. I have two files, with File 1 consisting of a single field of many thousands of records. I also have File 2 with two fields and many thousands of records. My goal is that when $1 of...

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

AWK - printing certain fields when field order changes in data file

Discussion started by: eric4

2. Shell Programming and Scripting

selective printing awk

Discussion started by: saint2006

3. Shell Programming and Scripting

AWK Matching Fields and Combining Files

Discussion started by: Michelangelo

4. Shell Programming and Scripting

AWK : Add Fields of lines with matching field

Discussion started by: DerSeb

5. Shell Programming and Scripting

Awk selective printing

Discussion started by: saint2006

6. Shell Programming and Scripting

Printing entire field, if at least one row is matching by AWK

Discussion started by: Chulamakuri

7. Shell Programming and Scripting

awk to combine all matching fields in input but only print line with largest value in specific field

Discussion started by: cmccabe

8. UNIX for Beginners Questions & Answers

Awk: matching multiple fields between 2 files

Discussion started by: beca123456

9. Shell Programming and Scripting

awk Selective printing of fields

Discussion started by: shunya

10. UNIX for Beginners Questions & Answers

awk for matching fields between files with repeated records

Discussion started by: jvoot