awk for comparing two files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk for comparing two files
# 1  
Old 10-17-2011
awk for comparing two files

so have file1 like this:

Code:
joe 123
jane 456

and then file2 like this:

Code:
123 left right
456 up down
joe ding dong
jane flip flop

what I need to do is compare col1 and col2 in file1 with col1 in file2 and generate a new file that has lines like this:

Code:
joe 123 ding dong left right
jane 456 flip flop up down

what I did so far is this:

Code:
# this grabs the 'left' value
awk 'NR==FNR{a[$1]=$2;next} {print $1,$2,a[$2]}' file2 file1 >> file3
# this grabs the 'right' value
awk 'NR==FNR{a[$1]=$3;next} {print $1,$2,$3,a[$2]}' file2 file3 >> file4

which generates this line:

joe 123 left right

but I am stuck on grabbing joe's 'ding dong' and also, am I doing this right with multiple awk lines, or can it be made more compact ?

Last edited by radoulov; 10-17-2011 at 04:50 PM.. Reason: Additional code tags!
# 2  
Old 10-17-2011
Code:
awk 'END {
  for (i = 0; ++i <= FNR;) 
    print r[i]
  }
NR == FNR {
  f[$1] = substr($0, index($0, $2)) 
  next
  }
{ 
  for (i = 0; ++i <= NF;)
    $i in f && r[FNR] = (r[FNR] ? r[FNR] : $0) \
      FS f[$i]
  }' file2 file1

This User Gave Thanks to radoulov For This Post:
# 3  
Old 10-17-2011
thanks. sorry for not putting the outputs in code tags as well.
where do I place an IF check to skip lines like this in file1:

Code:
joe joe (skip)
joe 123 (keep)
123 123 (skip)

# 4  
Old 10-17-2011
Code:
awk 'END {
  for (i = 0; ++i <= FNR;) 
    if (i in r) print r[i]
  }
NR == FNR {
  f[$1] = substr($0, index($0, $2)) 
  next
  }
$1 != $2 { 
  for (i = 0; ++i <= NF;)
    $i in f && r[FNR] = (r[FNR] ? r[FNR] : $0) \
      FS f[$i]
  }' file2 file1

This User Gave Thanks to radoulov For This Post:
# 5  
Old 10-17-2011
I think I'm starting to love awk ! it's awesome Smilie
# 6  
Old 10-17-2011
Another way.

Code:
awk 'NR==FNR{c[$1 FS $2];next} {a[$1]=a[$1] FS $2 FS $3}
    END{for ( i in c) {split(i,x,FS);if (x[1]!=x[2]) print i,a[x[1]],a[x[2]]}}' file1 file2

This User Gave Thanks to rdcwayx For This Post:
# 7  
Old 10-23-2011
thanks, rdcwayx, both solutions worked fine.

now I want to loop through both files and for each row I want to check if file1's 3th column equals file2's 4th column then append 5th column from file1 at the end of all the rows it matched in file2. example:

Code:
file1:
john doe 123 left right

file2:
address1 phone1 email1 123
address2 phone2 email2 123

output:
address1 phone1 email1 123 right
address2 phone2 email2 123 right

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Comparing two txt files with AWK

Hi, I need to compare two text files with awk. File1: ------- chr1 43815007 43815009 COSM19193 REF=TG;OBS=AA;ANCHOR=G AMPL495041 chr1 43815008 43815009 COSM18918 REF=G;OBS=T;ANCHOR=T AMPL495041 chr1 115256527 115256528 ... (6 Replies)
Discussion started by: RushiK
6 Replies

2. Shell Programming and Scripting

Comparing the matches in two files using awk when both files have their own field separators

I've two files with data like below: file1.txt: AAA,Apples,123 BBB,Bananas,124 CCC,Carrot,125 file2.txt: Store1|AAA|123|11 Store2|BBB|124|23 Store3|CCC|125|57 Store4|DDD|126|38 So,the field separator in file1.txt is a comma and in file2.txt,it is | Now,the output should be... (2 Replies)
Discussion started by: asyed
2 Replies

3. Shell Programming and Scripting

comparing 2 files with awk

Hi, I'm a new user in awk and i'm trying to compare two files to create a third one if some values match in both files. The first file has this content: s 45.960746365 _21_ AGT 2490 [21:0 22:0 s 45.980418496 _21_ AGT 2491 [21:0 22:0 s 46.000090627 _21_ AGT 2492 [21:0 22:0 s 47.906552206... (2 Replies)
Discussion started by: carlosoria
2 Replies

4. UNIX for Advanced & Expert Users

Comparing two files using awk

i have one file say file1 having many records.Each record contains 2000 characters.i have to compare 192-200 (stored as name)characters in this file from other file say file2 having name stored in 1-9 characters. after comparing i have to print the record from file1 in another file say file3 ... (3 Replies)
Discussion started by: sonam273
3 Replies

5. Shell Programming and Scripting

comparing two files using awk

hit brick wall while trying to knock up a script that will take values from the "lookup" file and look it up in the "target" file and return values that dont appear in "target" but do in "lookup". just knocked up something using bits from previous threads but theres gotta be something wrong... (13 Replies)
Discussion started by: jack.bauer
13 Replies

6. Shell Programming and Scripting

Comparing 2 input files -Awk

Compare 2 files and print the values input1 (c1 20 100 X_y10) along with one closest highest (c1 100 200 X_y10) and one lowest values (c1 10 15 X_y10) from input2 input1 c1 20 100 X_y10 input2 c1 5 10 X_y10 c1 10 15 X_y10 c1 100 200 X_y10 c1 200 300 X_y10 output ... (8 Replies)
Discussion started by: bumblebee_2010
8 Replies

7. Shell Programming and Scripting

comparing files with field using awk

hi, i have 1 files a.csv temp.out a.cvs looks like add,16390,180,674X,HALIFAX_COMMONS_X,902,497,902-209 add,16390,180,674X,HALIFAX_COMMONS_X,902,497,902-219 add,16390,180,674X,HALIFAX_COMMONS_X,902,497,902-220 add,16390,180,674X,HALIFAX_COMMONS_X,902,497,902-221 and temp.out looks... (1 Reply)
Discussion started by: raghavendra.cse
1 Replies

8. Shell Programming and Scripting

awk - comparing files

I've been trying to use awk to compare two files that have pretty much the same data in apart from certain lines where in one file a fields value has changed. I want to print the line from the first file and the changed line from the second file. At the moment, all I can get it to do is print the... (6 Replies)
Discussion started by: dbrundrett
6 Replies

9. Shell Programming and Scripting

Comparing two files using awk

Hi, I have 2 text files n1 and n2. cat n1 : -------- 1234567890 4444444444 cat n2 : --------- 1234567890 4444444444 7777777777 8888888888 they are some sample records. File n1 contains some lookup records, whereas file n2 contains some transaction records. I need to get the... (2 Replies)
Discussion started by: kanu_kanu
2 Replies

10. Shell Programming and Scripting

comparing two files using awk.

Hi All, a new bie to awk, How to compare substring of col1,file 1 with col2file2 and get file1contents+col3file2 as output. file1 ----- kumarfghh,23,12000,5000 rajakumar,24,14000,2500 rajeshchauhan,25,16000,2600 manoj,26,17000,2300 file 2 -------- 123,kumar,US, 123,sukumar,UK... (4 Replies)
Discussion started by: jerome Sukumar
4 Replies
Login or Register to Ask a Question