Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Continued trouble matching fields in different files and selective field printing ([g]awk)
Top Forums UNIX for Beginners Questions & Answers Continued trouble matching fields in different files and selective field printing ([g]awk) Post 303010716 by jvoot on Friday 5th of January 2018 11:38:56 PM
Old 01-06-2018
Please don't be sorry. I appreciate your help more than you know.

---------- Post updated at 08:38 PM ---------- Previous update was at 07:00 PM ----------

Don, if I may, your code help works terrific except I'm having issues when the records of $1 and $2 are repeated somewhere else in the field.

For example, there is a section in my data that looks like this portion of File1:

File1
Code:
PS012,007 DKJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13  13   2      -1      -1      -1    0  521     0
PS012,007 S>M                 0   2 -1 -1 -1  1 -1   -1 -1  0  2     3   2   0  -1      -1      -1      -1   -1   -1    -1
PS012,007 GBJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13   2  -1      -1      -1      -1    0  502     0
PS012,007 D                  -1   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  519     0
PS012,007 BXR                 0   1  0 17  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0
PS012,007 B                   0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
PS012,007 >R<                 0   2 -1 -1 -1  1 -1   -1 -1  0  1     3   2   5   2      -1      -1      -1    0  505     0
PS012,007 W                   0   6 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
PS012,007 DKJ                 0   1  6 18  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0

Here, there are two instances where $1=="PS12,007" and $2=="DKJ" (the first and last lines in the example, but these are not first and last in the actual file). However there is different data in fields $3-$23.

When I run the code you helped me with:
Code:
awk 'NR==FNR {A[$1,$2]=$0; next} ($2,$5) in A{print $1,A[$2,$5]}' file1 file2

with the following portion of File2:
Code:
120  PS012,007 ?          120 DKJ       
122  PS012,007 KSP        122 S>M       
123  PS012,007 ?          123 GBJ       
125  PS012,007 ?          125 D         
126  PS012,007 YRP        126 BXR       
127  PS012,007 B          127 B         
128  PS012,007 <LJL       128 >R<       
129  PS012,007 L          129 ?         
130  PS012,007 H          130 ?         
131  PS012,007 >RY        131 ?         
133  PS012,007 ?          133 W         
134  PS012,007 ZQQ        134 DKJ

I get this output:
Code:
120  PS012,007 DKJ                 0   1  6 18  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0
122  PS012,007 S>M                 0   2 -1 -1 -1  1 -1   -1 -1  0  2     3   2   0  -1      -1      -1      -1   -1   -1    -1
123  PS012,007 GBJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13   2  -1      -1      -1      -1    0  502     0
125  PS012,007 D                  -1   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  519     0
126  PS012,007 BXR                 0   1  0 17  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0
127  PS012,007 B                   0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
128  PS012,007 >R<                 0   2 -1 -1 -1  1 -1   -1 -1  0  1     3   2   5   2      -1      -1      -1    0  505     0
133  PS012,007 W                   0   6 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
134  PS012,007 DKJ                 0   1  6 18  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0

Here, fields $4-$24 are identical when fields $1 and $2 of File1 have a repeated string. However, fields $4-$24 should remain as they appear in File1 (i.e., fields $3-$23 in File1). The end result should look like this:
Code:
120  PS012,007 DKJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13  13   2      -1      -1      -1    0  521     0
122  PS012,007 S>M                 0   2 -1 -1 -1  1 -1   -1 -1  0  2     3   2   0  -1      -1      -1      -1   -1   -1    -1
123  PS012,007 GBJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13   2  -1      -1      -1      -1    0  502     0
125  PS012,007 D                  -1   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  519     0
126  PS012,007 BXR                 0   1  0 17  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0
127  PS012,007 B                   0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
128  PS012,007 >R<                 0   2 -1 -1 -1  1 -1   -1 -1  0  1     3   2   5   2      -1      -1      -1    0  505     0
133  PS012,007 W                   0   6 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
134  PS012,007 DKJ                 0   1  6 18  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0

I've attempted various code variations that take what you had helped me with and integrates it with count and/or loop functions in awk. None of these have been successful.
Last edited by jvoot; 01-06-2018 at 12:53 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

AWK - printing certain fields when field order changes in data file

I'm hoping someone can help me on this. I have a data file that greatly simplified might look like this: sec;src;dst;proto 421;10.10.10.1;10.10.10.2;tcp 426;10.10.10.3;10.10.10.4;udp 442;10.10.10.5;10.10.10.6;tcp sec;src;fac;dst;proto 521;10.10.10.1;ab;10.10.10.2;tcp... (3 Replies)
Discussion started by: eric4
3 Replies

2. Shell Programming and Scripting

selective printing awk

Hi there my file looks like this 1 a b c d e f 2 a b b c d e f f g h e t t 3 a c b d e f 4 a b c i want to print the line which has the fields containing ONLY a b c, in this case the line 4. How can i awk it !!!? Many Thanks in advance! (8 Replies)
Discussion started by: saint2006
8 Replies

3. Shell Programming and Scripting

AWK Matching Fields and Combining Files

Hello! I am writing a program to run through two large lists of data (~300,000 rows), find where rows in one file match another, and combine them based on matching fields. Due to the large file sizes, I'm guessing AWK will be the most efficient way to do this. Overall, the input and output I'm... (5 Replies)
Discussion started by: Michelangelo
5 Replies

4. Shell Programming and Scripting

AWK : Add Fields of lines with matching field

Dear All, I would like to add values of a field, if the lines match in a certain field. Then I would like to divide the sum though the number of lines that have a matched field. This is the Input: Input: Test1 5 Test1 10 Test2 2 Test2 5 Test2 13 Test3 4 Output: Test1 7.5 Test1 7.5... (6 Replies)
Discussion started by: DerSeb
6 Replies

5. Shell Programming and Scripting

Awk selective printing

Hi, i need help to print number from different field INPUT: Student1 10 20 Student2 30 40 Student3 50 60 Student4 70 80 Desired Output: 1 20-30 2 40-50 3 60-70 Thank you! (5 Replies)
Discussion started by: saint2006
5 Replies

6. Shell Programming and Scripting

Printing entire field, if at least one row is matching by AWK

Dear all, I have been trying to print an entire field, if the first line of the field is matching. For example, my input looks something like this. aaa ddd zzz 123 987 126 24 0.650 985 354 9864 0.32 0.333 4324 000 I am looking for a pattern,... (5 Replies)
Discussion started by: Chulamakuri
5 Replies

7. Shell Programming and Scripting

awk to combine all matching fields in input but only print line with largest value in specific field

In the below I am trying to use awk to match all the $13 values in input, which is tab-delimited, that are in $1 of gene which is just a single column of text. However only the line with the greatest $9 value in input needs to be printed. So in the example below all the MECP2 and LTBP1... (0 Replies)
Discussion started by: cmccabe
0 Replies

8. UNIX for Beginners Questions & Answers

Awk: matching multiple fields between 2 files

Hi, I have 2 tab-delimited input files as follows. file1.tab: green A apple red B apple file2.tab: apple - A;Z Objective: Return $1 of file1 if, . $1 of file2 matches $3 of file1 and, . any single element (separated by ";") in $3 of file2 is present in $2 of file1 In order to... (3 Replies)
Discussion started by: beca123456
3 Replies

9. Shell Programming and Scripting

awk Selective printing of fields

Hi Gurus, I have following input file. I tried multiple awk combinations to print selected columns without success. HEX ID Name ver FLRGT Start Time Total Shared End Date ----- -------- --- ------ ------------------------ -------------- -------... (4 Replies)
Discussion started by: shunya
4 Replies

10. UNIX for Beginners Questions & Answers

awk for matching fields between files with repeated records

Hello all, I am having trouble with what should be an easy task, but seem to be missing something fundamental. I have two files, with File 1 consisting of a single field of many thousands of records. I also have File 2 with two fields and many thousands of records. My goal is that when $1 of... (2 Replies)
Discussion started by: jvoot
2 Replies
All times are GMT -4. The time now is 01:06 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy