AWK match $1 $2 pattern in file 1 to $1 $2 pattern in file2


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting AWK match $1 $2 pattern in file 1 to $1 $2 pattern in file2
# 1  
Old 10-06-2011
AWK match $1 $2 pattern in file 1 to $1 $2 pattern in file2

Hi, I have 2 files that I have modified to basically match each other, however I want to determine what (if any) line in file 1 does not exist in file 2. I need to match column $1 and $2 as a single string in file1 to $1 and $2 in file2 as these two columns create a match.

I'm stuck in an AWK command and can't get any further. Below is an example of my 2 files. Note the difference in the 2 files: file1 has TEST in $1 on a line and has an additional line at the bottom where $1 match in both files but $2 does not.

file1
Code:
 
AO|IO.SYS.123|ABCXYZ
AR|IO.SYS.XYZ|ABC123
AT|IO.SYS.ABC|ABC321
TEST|IO.SYS.ABC|444555
AW|IO.SYS.XXX|123456
AZ|IO.SYS.ABC|222555
BB|IO.SYS.ABC|ABCABC
BC|IO.SYS.ABC|111222
BI|IO.SYS.ABC|8889BC
BR|IO.SYS.ABC|ABCVER
BR|IO.SYS.XYZ|ABCVER

file2
Code:
 
AO|IO.SYS.123|ABCXYZ
AR|IO.SYS.XYZ|ABC123
AT|IO.SYS.ABC|ABC321
AW|IO.SYS.XXX|123456
AZ|IO.SYS.ABC|222555
BB|IO.SYS.ABC|ABCABC
BC|IO.SYS.ABC|111222
BI|IO.SYS.ABC|8889BC
BR|IO.SYS.ABC|ABCVER

I can't figure out how to match both $1 and $2 as 1 string in both files. $1 in file1 might exist in file2 but $1 and $2 must match each other. If not determine which line does not exist.
# 2  
Old 10-06-2011
Is the second column supposed to be unique for each line?
# 3  
Old 10-06-2011
Code:
awk -F\| 'NR == FNR {
  _[$1, $2]; next
  }
!(($1, $2) in _ )' file2 file1

# 4  
Old 10-06-2011
I'm curious how that works.
# 5  
Old 10-06-2011
It builds an associative array indexed by the concatenation of the first field, a special character (the current value of SUBSEP, default \034) and the second field, then it checks if such key is present (while reading the next file and constructing the key dynamically ...). Of course, I could be missing something ...
To the OP: does this code produce the expected output?
# 6  
Old 10-06-2011
You can use grep as well
Code:
grep -F -v -f file2 file1

Regards
Peasant.
# 7  
Old 10-06-2011
yes i tested it and it seems to work perfect. thanks!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to update value based on pattern match in another file

In the awk, thanks you @RavinderSingh13, for the help in below, hopefully it is close as I am trying to update the value in $12 of the tab-delimeted file2 with the matching value in $1 of the space delimeted file1. I have added comments for each line as well. Thank you :). awk awk '$12 ==... (10 Replies)
Discussion started by: cmccabe
10 Replies

2. Shell Programming and Scripting

awk command to get file content until 2 occurrence of pattern match

AWK command to get file content until 3 occurrence of pattern match, INPUT FILE: JMS_BODY_FIELD:JMSText = <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <custOptIn xmlns="http://com/walm/ta/cu/ccs/xml2"> <person>Romi</person> <appName>SAP</appName> </custOptIn> ... (4 Replies)
Discussion started by: prince1987
4 Replies

3. Shell Programming and Scripting

awk to combine lines from line with pattern match to a line that ends in a pattern

I am trying to combine lines with these conditions: 1. First line starts with text of "libname VALUE db2 datasrc" where VALUE can be any text. 2. If condition1 is met then continue to combine lines through a line that ends with a semicolon. 3. Ignore case when matching patterns and remove any... (5 Replies)
Discussion started by: Wes Kem
5 Replies

4. Shell Programming and Scripting

Rearrange or replace only the second line after pattern match or pattern match

Im using the command below , but thats not the output that i want. it only prints the odd and even numbers. awk '{if(NR%2){print $0 > "1"}else{print $0 > "2"}}' Im hoping for something like this file1: Text hi this is just a test text1 text2 text3 text4 text5 text6 Text hi... (2 Replies)
Discussion started by: invinzin21
2 Replies

5. Shell Programming and Scripting

awk Pattern Match One File to Another

I want to read from file 1 and pattern match in file two and print field two from the next line. File 1: user1 user2 user3 File 2: name=user1 gud=12345 name=user2 gud=32456 I have this pattern hardcoded but can't work out how to pass file 1 to the pattern match: (6 Replies)
Discussion started by: u20sr
6 Replies

6. UNIX for Dummies Questions & Answers

Match Pattern after certain pattern and Print words next to Pattern

Hi experts , im new to Unix,AWK ,and im just not able to get this right. I need to match for some patterns if it matches I need to print the next few words to it.. I have only three such conditions to match… But I need to print only those words that comes after satisfying the first condition..... (2 Replies)
Discussion started by: 100bees
2 Replies

7. Shell Programming and Scripting

Awk to match a pattern and perform a search after the first pattern

Hello Guyz I have been following this forum for a while and the solutions provided are super useful. I currently have a scenario where i need to search for a pattern and start searching by keeping the first pattern as a baseline ABC DEF LMN EFG HIJ LMN OPQ In the above text i need to... (8 Replies)
Discussion started by: RickCharles
8 Replies

8. Shell Programming and Scripting

AWK: Pattern match between 2 files, then compare a field in file1 as > or < field in file2

First, thanks for the help in previous posts... couldn't have gotten where I am now without it! So here is what I have, I use AWK to match $1 and $2 as 1 string in file1 to $1 and $2 as 1 string in file2. Now I'm wondering if I can extend this AWK command to incorporate the following: If $1... (4 Replies)
Discussion started by: right_coaster
4 Replies

9. Shell Programming and Scripting

Use to awk to match pattern, and print the pattern

Hi, I know how to use awk to search some expressions like five consecutive numbers, , this is easy. However, how do I make awk print the pattern that is been matched? For example: input: usa,canada99292,japan222,france59664,egypt223 output:99292,59664 (6 Replies)
Discussion started by: grossgermany
6 Replies

10. UNIX for Advanced & Expert Users

print contents of file2 for matching pattern in file1 - AWK

File1 row is same as column 2 in file 2. Also file 2 will either start with A, B or C. And 3rd column in file 2 is always F2. When column 2 of file 2 matches file1 column, print all those rows into a separate file. Here is an example. file 1: 100 103 104 108 file 2: ... (6 Replies)
Discussion started by: i.scientist
6 Replies
Login or Register to Ask a Question