Match Strings between two files, print portions of each file together when matched ([g]awk)


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Match Strings between two files, print portions of each file together when matched ([g]awk)
# 1  
Old 06-27-2017
Match Strings between two files, print portions of each file together when matched ([g]awk)

I have two files and desire to use the strings from $1 of file 1 (file1.txt) as search criteria to find matches in $2 of file 2 (file2.txt). If matches are found I want to output the entire line of file 2 (file2.txt) followed by fields $2-$11 of file 1 (file1.txt). I can find the matches, I cannot seem to get both files to print correctly. The data looks thusly:

file1.txt
Code:
  1 >CR=        2 -1 -1 -1  5  0 -1 -1  3  2
  2 H           0 -1 -1 -1 -1 -1 -1 -1 -1 -1
  3 >JC         2 -1 -1 -1  1  0 -1 -1  1  2
  4 
  5 >CR         6 -1 -1 -1 -1 -1 -1 -1 -1 -1

file2.txt
Code:
  1  2 >CR=                 2 VWB
  2  2 >JC                  2 GBR
  3                           
  4  6 >CR                  6 D

Desired Output:
Code:
1 2 >CR=                 2 VWB    2 -1 -1 -1  5  0 -1 -1  3  2
2 2 >JC                  2 GBR    2 -1 -1 -1  1  0 -1 -1  1  2
3                          
4 6 >CR                  6 D      6 -1 -1 -1 -1 -1 -1 -1 -1 -1

I can find the matches between the two files and respective fields easy enough and print the lines of file1.txt with:
Code:
awk 'NR==FNR{a[$1];next}$2 in a{print $0}' file1.txt file2.txt

Yet, when it comes to printing the desired fields from the second file after the lines of the first file, this is where I run aground.

Thanks in advance for your help and if I may, I'm wondering if the respondent might offer a word or two by way of explanation for as a beginner I seem to often run into difficulty when attempting to use arrays in awk.
# 2  
Old 06-27-2017
Hello jvoot,

Could you please try following and let me know if this helps you.
Code:
awk 'FNR==NR{q=$2;$1=$2="";A[q]=$0;next} ($3 in A){print $0,A[$3]}'  Input_file1   Input_file2

Thanks,
R. Singh
# 3  
Old 06-27-2017
Thanks so much for this R. Singh. Unfortunately, it didn't seem to do the trick. For example, line 4 of 'Desired Output' should be:

Code:
4 6 >CR                  6 D      6 -1 -1 -1 -1 -1 -1 -1 -1 -1

But in the code you offered, it is:
Code:
6 >CR                  6 D   -1 -1 -1 -1 -1 -1 -1 -1 -1

As I look through $5 of the output of your code, which is supposed to constitute $2 of file1.txt, I see values there that do not exist in the entirety $2 for the entire file (i.e., $5 of output has a value that is not contained in $2 of file1.txt at all).

Thanks so much for the attempt though!
# 4  
Old 06-28-2017
Quote:
Originally Posted by jvoot
Output' should be:
Code:
4 6 >CR                  6 D      6 -1 -1 -1 -1 -1 -1 -1 -1 -1

But in the code you offered, it is:
Code:
6 >CR                  6 D   -1 -1 -1 -1 -1 -1 -1 -1 -1

Hello jvoot,

I am seeing output from my code which you are expecting only, see following.
Code:
awk 'FNR==NR{q=$2;$1=$2="";A[q]=$0;next} ($3 in A){print $0,A[$3]}' file1  file2
1  2 >CR=                 2 VWB   2 -1 -1 -1 5 0 -1 -1 3 2
2  2 >JC                  2 GBR   2 -1 -1 -1 1 0 -1 -1 1 2
3
4  6 >CR                  6 D   6 -1 -1 -1 -1 -1 -1 -1 -1 -1

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
# 5  
Old 06-28-2017
Interesting. I just ran it again, and it is it doesn't seem to be working. Indeed, the four lines of output you posted are correct, but the four that I get are off. I'm going to have to do some more investigating.

Here is my output for
Code:
awk 'FNR==NR{q=$2;$1=$2="";A[q]=$0;next} ($3 in A){print $0,A[$3]}' file1 file2

Code:
 2 >CR=                 2 VWB   -1 -1 -1 1 0 -1 -1 1 0
 2 >JC                  2 GBR   -1 -1 -1 1 0 -1 -1 1 0
                            
 6 >CR                  6 D   -1 -1 -1 -1 -1 -1 -1 -1 -1

---------- Post updated at 08:54 PM ---------- Previous update was at 08:41 PM ----------

Ah, R. Singh! I think I figured it out! In your code:

Code:
awk 'FNR==NR{q=$2;$1=$2="";A[q]=$0;next} ($3 in A){print $0,A[$3]}' file1 file2

You have both fields $1 and $2, set to null ("") and therefore it is just repeating one of the fields when it goes to print. If only $1 of file1 is set to null ($1="") then it works as you say for the lines of output here.

Code:
awk 'FNR==NR{q=$2;$1="";A[q]=$0;next} ($3 in A){print $0,A[$3]}' file1 file2

 2 >CR=                 2 VWB  2 -1 -1 -1 1 0 -1 -1 1 0
 2 >JC                  2 GBR  2 -1 -1 -1 1 0 -1 -1 1 0
                           
 6 >CR                  6 D  6 -1 -1 -1 -1 -1 -1 -1 -1 -1

However, even at that, as I scan through the rest of the output there are other lines not showing the correct values.

Last edited by jvoot; 06-28-2017 at 01:05 AM..
# 6  
Old 06-28-2017
A small correction to RavinderSingh13's yields
Code:
awk '
FNR==NR         {q    = $1
                 $1   = $2 = ""
                 A[q] = $0
                 next
                }
                {print $0, A[$2]
                }
'  file1 file2
  1  2 >CR=                 2 VWB   0 -1 -1 -1 -1 -1 -1 -1 -1 -1
  2  2 >JC                  2 GBR   0 -1 -1 -1 -1 -1 -1 -1 -1 -1
  3                            
  4  6 >CR                  6 D

which is as close to the requested output as you can get. Please note that the desired output given in post#1 does NOT satisfy the specification as there's NO match of file2's line 4's 6 in file1!
This User Gave Thanks to RudiC For This Post:
# 7  
Old 06-29-2017
I see what was wrong. In my example data I foolishly included the line numbers from my file. Once I adjusted fields for this error it worked flawlessly.

Thank you so much RudiC and R. Singh.

If it is not too much trouble, would someone be kind enough to explain this code for me? I'm tracking fairly well but got confused when variable "q" is assigned to $2 but then is included in array A.

Last edited by jvoot; 06-29-2017 at 02:52 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Use strings from nth field from one file to match strings in entire line in another file, awk

I cannot seem to get what should be a simple awk one-liner to work correctly and cannot figure out why. I would like to use patterns from a specific field in one file as regex to search for matching strings in the entire line ($0) of another file. I would like to output the lines of File2 which... (1 Reply)
Discussion started by: jvoot
1 Replies

2. UNIX for Beginners Questions & Answers

Match Fields between two files, print portions of each file together when matched in ([g]awk)'

I've written an awk script to compare two fields in two different files and then print portions of each file on the same line when matched. It works reasonably well, but every now and again, I notice some errors and cannot seem to figure out what the issue may be and am turning to you for help. ... (2 Replies)
Discussion started by: jvoot
2 Replies

3. Shell Programming and Scripting

awk to print fields that match using conditions and a default value for non-matching in two files

Trying to use awk to match the contents of each line in file1 with $5 in file2. Both files are tab-delimited and there may be a space or special character in the name being matched in file2, for example in file1 the name is BRCA1 but in file2 the name is BRCA 1 or in file1 name is BCR but in file2... (6 Replies)
Discussion started by: cmccabe
6 Replies

4. Shell Programming and Scripting

Regex: print matched line and exact pattern match

Hi experts, I have a file with regexes which is used for automatic searches on several files (40+ GB). To do some postprocessing with the grep result I need the matching line as well as the match itself. I know that the latter could be achieved with grep's -o option. But I'm not aware of a... (2 Replies)
Discussion started by: stresing
2 Replies

5. Shell Programming and Scripting

Print only lines where fields concatenated match strings

Hello everyone, Maybe somebody could help me with an awk script. I have this input (field separator is comma ","): 547894982,M|N|J,U|Q|P,98,101,0,1,1 234900027,M|N|J,U|Q|P,98,101,0,1,1 234900023,M|N|J,U|Q|P,98,54,3,1,1 234900028,M|H|J,S|Q|P,98,101,0,1,1 234900030,M|N|J,U|F|P,98,101,0,1,1... (2 Replies)
Discussion started by: Ophiuchus
2 Replies

6. Shell Programming and Scripting

Print strings that match pattern with awk

I have a file with many lines which contain strings like .. etc. But with no rule regarding field separators or anything else. I want to print ONLY THE STRING from each line , not the entire line !!! For example from the lines : Flow on service executed with success in . Performances... (5 Replies)
Discussion started by: black_fender
5 Replies

7. Shell Programming and Scripting

match two key columns in two files and print output (awk)

I have two files... file1 and file2. Where columns 1 and 2 of file1 match columns 1 and 2 of file2 I want to create a new file that is all file1 + columns 3 and 4 of file2 :b: Many thanks if you know how to do this.... :b: file1 31-101 106 0 92 31-101 106 29 ... (2 Replies)
Discussion started by: pelhabuan
2 Replies

8. Shell Programming and Scripting

awk strings search + print next column after match

Hi, I have a file filled with search strings which have a blank in between and look like this: S. g. Ehr. o. Jg. v. d. Chijs g. Ehr. Now i would like to search for the strings and it also shall return the next column after the match. awk -v FILE="search_strings.txt" 'BEGIN {... (10 Replies)
Discussion started by: sdf
10 Replies

9. Shell Programming and Scripting

Match a line in File 1 with Column in File 2 and print whole line in file 2 when matched

Hi Experts, I am very new to scripting and have a prb since few days and it is urgent to solve so much appreciated if u help me. i have 2 files file1.txt 9647810043118 9647810043126 9647810043155 9647810043161 9647810043166 9647810043185 9647810043200 9647810043203 9647810043250... (22 Replies)
Discussion started by: mustafa.abdulsa
22 Replies

10. Shell Programming and Scripting

Strings from one file which exactly match to the 1st column of other file and then print lines.

Hi, I have two files. 1st file has 1 column (huge file containing ~19200000 lines) and 2nd file has 2 columns (small file containing ~6000 lines). ################################# huge_file.txt a a ab b ################################## small_file.txt a 1.5 b 2.5 ab ... (4 Replies)
Discussion started by: AshwaniSharma09
4 Replies
Login or Register to Ask a Question