Visit Our UNIX and Linux User Community


Awk script / comparing two files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Awk script / comparing two files
# 1  
Old 10-21-2009
Question Awk script / comparing two files

Goal: To combine the data from two files into one file.

File1 = 11 fields, the first field is the unique key
File2 = 2 fields, the first field is the unique key

What I want to do is match File2:column1 with File1:column1 and if it matches, to add the data from File2:column2 to the matching line in File1 (in essence, creating an 12th column).

Field delimiter = pipe (|)

Sample data from File1:
3065462|245|1|Z682.4 .S89 B33 2007|3531301|0|10/20/2009|3531301|LIBR|CENTRAL|Complete.
3066405|245|1|jlk|3532264|0|10/20/2009|3532264|LIBR|CENTRAL|Energy.

Sample data from File2:
3065462|Westport, Conn. ; London : Libraries Unlimited, 2007.
3066405|Washington : U.S. Govt. Print. Off., 1976.

Desired output:
3065462|245|1|Z682.4 .S89 B33 2007|3531301|0|10/20/2009|3531301|LIBR|CENTRAL|Complete.|Westport, Conn. ; London : Libraries Unlimited, 2007.
3066405|245|1|jlk|3532264|0|10/20/2009|3532264|LIBR|CENTRAL|Energy.|Washington : U.S. Govt. Print. Off., 1976.

I've tried this, with no luck:
Code:
awk -F"|" 'BEGIN {
while ((getline < "file2") > 0) 
f2array[$1] = $1 
OFS="|"} 
{if (f2array[$1]) 
print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,f2array[$2] 
}' file1 > results-all

I've also tried changing $1 to $0, but that didn't work either. I've actually tried several things over the past several days, but either I get nothing or I get some kind of awk error:

awk: syntax error near line 4
awk: illegal statement near line 4
awk: syntax error near line 13
awk: bailing out near line 13

(from this script -
Code:
#!/bin/awk -f
BEGIN {
FS = "|"
while ((getline < ARGV[0]) > 0)
 f2array[$1] = $1
OFS="|"}

{if (f2array[$1])
 print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,f2array[$2]
else
 print $1 " not listed in resultst" > "unmatched"
} ARGV[1]

./test.awk resultsp resultst)

Last edited by jmcgranahan; 10-21-2009 at 11:22 PM..
# 2  
Old 10-21-2009
Hi,

Try this:

Code:
awk -F"|" -v OFS="|" 'FNR==NR{a[$1]=$2;next}a[$1]{$12=a[$1]}1' file2 file1

And please edit your post and use the code tags.

Last edited by ripat; 10-21-2009 at 06:16 PM.. Reason: fixed
# 3  
Old 10-22-2009
MySQL

Well, I tried your suggestion and still no dice:

# awk -F"|" -v OFS="|" 'FNR==NR{a[$1]=$2;next}a[$1]{$12=a[$1]}1' file2 file1
awk: syntax error near line 1
awk: bailing out near line 1

so I tried the same command with nawk and that worked! Interesting. Thanks for your help!

Previous Thread | Next Thread
Test Your Knowledge in Computers #582
Difficulty: Medium
A buffered input or output stream is one in which data that is read or written is actually stored in memory in a buffer until such a time as the buffer is flushed and the accumulated data is passed to or from the actual file.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Comparing the two files using awk script

Hi all, Can you please help me to find out that where is the problem in my script or either my way of writing the shell command on the prompt is not right? Actually, I want to compare the second column "$1" of the file "t1" with all the columns of second file "t2", if there is a match then the... (2 Replies)
Discussion started by: coder83
2 Replies

2. Shell Programming and Scripting

awk for comparing two files

so have file1 like this: joe 123 jane 456 and then file2 like this: 123 left right 456 up down joe ding dong jane flip flop what I need to do is compare col1 and col2 in file1 with col1 in file2 and generate a new file that has lines like this: joe 123 ding dong left right jane... (11 Replies)
Discussion started by: Jaymz
11 Replies

3. Shell Programming and Scripting

Comparing the matches in two files using awk when both files have their own field separators

I've two files with data like below: file1.txt: AAA,Apples,123 BBB,Bananas,124 CCC,Carrot,125 file2.txt: Store1|AAA|123|11 Store2|BBB|124|23 Store3|CCC|125|57 Store4|DDD|126|38 So,the field separator in file1.txt is a comma and in file2.txt,it is | Now,the output should be... (2 Replies)
Discussion started by: asyed
2 Replies

4. Shell Programming and Scripting

comparing 2 files with awk

Hi, I'm a new user in awk and i'm trying to compare two files to create a third one if some values match in both files. The first file has this content: s 45.960746365 _21_ AGT 2490 [21:0 22:0 s 45.980418496 _21_ AGT 2491 [21:0 22:0 s 46.000090627 _21_ AGT 2492 [21:0 22:0 s 47.906552206... (2 Replies)
Discussion started by: carlosoria
2 Replies

5. UNIX for Advanced & Expert Users

Comparing two files using awk

i have one file say file1 having many records.Each record contains 2000 characters.i have to compare 192-200 (stored as name)characters in this file from other file say file2 having name stored in 1-9 characters. after comparing i have to print the record from file1 in another file say file3 ... (3 Replies)
Discussion started by: sonam273
3 Replies

6. Shell Programming and Scripting

comparing two files using awk

hit brick wall while trying to knock up a script that will take values from the "lookup" file and look it up in the "target" file and return values that dont appear in "target" but do in "lookup". just knocked up something using bits from previous threads but theres gotta be something wrong... (13 Replies)
Discussion started by: jack.bauer
13 Replies

7. Shell Programming and Scripting

awk script issue : comparing two files with a pattern

File 1 ################################################################# pma.zcal.iop_pma_zcal_cntl (2710.080 115.200) pma.lanea23.rx0.cntl (696.960 844.800) pma.lanea67.rx0.cntl (1733.760 844.800) pma.zcal.iop_pma_zcal_cust (2280.960 115.200)... (1 Reply)
Discussion started by: jaita
1 Replies

8. Shell Programming and Scripting

awk - comparing files

I've been trying to use awk to compare two files that have pretty much the same data in apart from certain lines where in one file a fields value has changed. I want to print the line from the first file and the changed line from the second file. At the moment, all I can get it to do is print the... (6 Replies)
Discussion started by: dbrundrett
6 Replies

9. Shell Programming and Scripting

Comparing two files using awk

Hi, I have 2 text files n1 and n2. cat n1 : -------- 1234567890 4444444444 cat n2 : --------- 1234567890 4444444444 7777777777 8888888888 they are some sample records. File n1 contains some lookup records, whereas file n2 contains some transaction records. I need to get the... (2 Replies)
Discussion started by: kanu_kanu
2 Replies

10. Shell Programming and Scripting

comparing two files using awk.

Hi All, a new bie to awk, How to compare substring of col1,file 1 with col2file2 and get file1contents+col3file2 as output. file1 ----- kumarfghh,23,12000,5000 rajakumar,24,14000,2500 rajeshchauhan,25,16000,2600 manoj,26,17000,2300 file 2 -------- 123,kumar,US, 123,sukumar,UK... (4 Replies)
Discussion started by: jerome Sukumar
4 Replies

Featured Tech Videos