Compare two files using awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare two files using awk
# 1  
Old 05-26-2009
Compare two files using awk

Hi. I'm new to awk and have searched for a solution to my problem, but haven't found the right answer yet. I have two files that look like this:

file1
Delete,3105551234
Delete,3105551236
Delete,5625559876
Delete,5625556789
Delete,5625553456
Delete,5625551234
Delete,5625556956
Delete,5625556643
Delete,6265552486
Delete,6265559365
Add,7755559833
Add,9515550087


file2
93,170334,0,-1,-1,,AAA,,5625556643,6465550987,,,-1,,581,93,-1
94,170335,0,-1,-1,,AAA,,7145550167,6465550987,,,-1,,581,93,-1
107,170239,0,-1,-1,,AAA,,6265559999,6465550987,,,-1,,581,93,-1
109,170240,0,-1,-1,,AAA,,5205558723,6465550987,,,-1,,581,93,-1
110,170241,0,-1,-1,,AAA,,3105551236,6465550987,,,-1,,581,93,-1
111,170348,0,-1,-1,,AAA,,6195550178,6465550987,,,-1,,581,93,-1
114,170256,0,-1,-1,,AAA,,5625559876,6465550987,,,-1,,581,93,-1
118,170336,0,-1,-1,,AAA,,3105551234,6465550987,,,-1,,581,93,-1
119,170337,0,-1,-1,,AAA,,5125559812,6465550987,,,-1,,581,93,-1
120,170338,0,-1,-1,,AAA,,5125559083,6465550987,,,-1,,581,93,-1
121,101,1,-1,-1,,AAA,,,2135559126,,,-1,,0,85,-1
122,170339,0,-1,-1,,AAA,,5625559067,6465550987,,,-1,,581,93,-1
125,999996,1,-1,-1,,AAA,,,6265559365,,,-1,,0,2561,-1
127,170340,0,-1,-1,,AAA,,5625551234,6465550987,,,-1,,581,93,-1
128,170341,0,-1,-1,,AAA,,5625559148,6465550987,,,-1,,581,93,-1
129,170342,0,-1,-1,,AAA,,5625556789,6465550987,,,-1,,581,93,-1
130,170343,0,-1,-1,,AAA,,5625559210,6465550987,,,-1,,581,93,-1
133,100,1,-1,-1,,AAA,,,6265552486,,,-1,,0,85,-1
134,170344,0,-1,-1,,AAA,,5625553456,6465550987,,,-1,,581,93,-1
135,170345,0,-1,-1,,AAA,,7605559809,6465550987,,,-1,,581,93,-1
137,170257,0,-1,-1,,AAA,,5625556956,6465550987,,,-1,,581,93,-1

I would like to look at file1 and any entry that has "Delete" in $1, look for $2 (from file1) in file2. Then, create a third file, file3, with "D,"$1 of file2. So, the output with the above examples would look like this:

file3
D,93
D,110
D,114
D,118
D,125
D,127
D,129
D,133
D,134
D,137

I hope I'm making sense. Any help would be appreciated. Thanks.
# 2  
Old 05-26-2009
Not to quibble - but you are not clear. Your example does not match what you said.
take
114,170256,0,-1,-1,,AAA,,5625559876,6465550987,,,-1,,581,93,-1
and
Delete,5625559876

This means 'do not print' the 114,...... line.

Your output
D,114

has the 114 line in it. Several other lines are like this. Did you mean the reverse of what you said?
# 3  
Old 05-26-2009
He means something like:

Code:
awk -F, 'NR==FNR && /^D/ {a[$2]++;next}
$9 in a || $10 in a {print "D," $1}' file1 file2

# 4  
Old 05-26-2009
Sorry about that. file1 is a list of numbers that need to be deleted or added. file2 is a list of current numbers and corresponding information. I want file3 to be just the "D," along with the first column of file2 associated with the number marked for deletion in file1.

I tried the script Franklin posted, but I got "syntax error near line 2". I forgot to mention I'm using Solaris 8 if that makes a difference. Thanks.
# 5  
Old 05-26-2009
Use nawk or /usr/xpg4/bin/awk on Solaris.

Regards
# 6  
Old 05-26-2009
Yes, nawk worked, thank you very much. I was wondering, if you didn't mind, if you could breakdown the script so I can understand exactly how it's working? I'd like to learn as much of this as I can. Thanks.
# 7  
Old 05-26-2009
Quote:
Originally Posted by paul.o
Yes, nawk worked, thank you very much. I was wondering, if you didn't mind, if you could breakdown the script so I can understand exactly how it's working? I'd like to learn as much of this as I can. Thanks.
Code:
awk -F, 'NR==FNR && /^D/ {a[$2]++;next}
$9 in a || $10 in a {print "D," $1}' file1 file2

Here we go:

Code:
awk -F,

Set field separator

Code:
NR==FNR && /^D/

If we read the 1st file and the line starts with a "D"

Code:
{a[$2]++;next}

Set array a with the 2nd field as index and read the next line

Code:
$9 in a || $10 in a {print "D," $1}'

If the 9th or the 10th field exists as an index of the array a in the 2nd file print "D," and the 1st field.


Regards
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[awk] Compare two files

HI!! I am trying to compare two files using AWK but I have some problems. I need to count how many times letters are used in two texts. This is my script { long=length($0) for (i=1;i<=long;i++) { aux=substr($0,i,1) if ( aux != " " && aux != "" ) ... (7 Replies)
Discussion started by: ettore8888
7 Replies

2. Shell Programming and Scripting

awk compare files

I have a below requirement and trying to compare the files using awk File 1 - Already stored on a prev day id | text | email id --------------------------------- 89564|this is line 1 | xyz@sample.txt 985384|this is line 2 | abc@sample.txt 657342|this is line 3 |... (3 Replies)
Discussion started by: rakesh_411
3 Replies

3. Shell Programming and Scripting

Compare 2 files, awk maybe?

I have 2 files, file1: alfa numbers numbers vita numbers numbers gama numbers numbers delta numbers numbers epsilon numbers numbers zita numbers numbers ... file2: 'zita' keepnumbers keepnumbers keepnumbers 'gama' keepnumbers keepnumbers keepnumbers 'misc' ... (11 Replies)
Discussion started by: phaethon
11 Replies

4. HP-UX

Awk compare two files

Hi guys, I have 2 files: File1 ABC|2203|115.50 ABC|2288|328.12 ABC|2289|611.09 ABC|2290|698 DEF|1513|721.3 DEF|1514|40 DEF|1515|5 File2 ABC|2288|328.12 ABC|2289|666.08 ABC|2290|698.00 DEF|1513|721.30 (3 Replies)
Discussion started by: Eduardo Aceves
3 Replies

5. Shell Programming and Scripting

Compare files using awk

Please help me to compare two files and remove the items in file2 from file1 file 1:delimited using pipe(|) file1 00012|Description - 1|||||AA12345|1|AB12345|2|2012/06/03 AB123|Description - 2|||||AA12345|3|ZA11111|4|2012/06/04 11111|Description - 3|||||AP00012|1|AB12345|2|2012/06/03... (8 Replies)
Discussion started by: Mary James
8 Replies

6. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

7. Shell Programming and Scripting

Compare two files with awk

Hello, I have a script which extracts the values from a csv file when a specific date is entered : #!/bin/sh awk 'BEGIN{printf("Entrez la date : "); getline date < "-"} $0 ~ date {f=1;print;next} /^{2}\//{f=0} f' file1.csv This script gives me a number of lines with different values. ... (6 Replies)
Discussion started by: freyr
6 Replies

8. UNIX for Dummies Questions & Answers

Using AWK to compare 2 files

Hi How can I use awk to compare specific columns in 2 files and print the difference. I currently have this: BEGIN { OFS = FS = "," } NR == FNR { b = $3 next } { e = "" for (x in b) { if (match ($1, x)) { if (RSTART == 1 && RLENGTH > length(e)) { e=x (2 Replies)
Discussion started by: ladyAnne
2 Replies

9. Shell Programming and Scripting

compare two files using awk

Hi, I want to compare two files using awk and write an output based on if the records matched. Both the files are space delimitted. File A: 8351 00000000000636 2009044 -00001.000 8351 00000000000637 2009044 -00002.000 8351 00000000000638 2009044 -00001.000 8351 00000000000640... (7 Replies)
Discussion started by: gpaulose
7 Replies

10. Shell Programming and Scripting

awk compare 2 files

Hi i hope some awk gurus here can help me.. here is what i need i have 2 files: File1 152445 516532 405088.pdf 152445 516533 405089.pdf 152491 516668 405153.jpg 152491 520977 408779.jpg 152491 0 409265.pdf File2 516532 /tmp/MainStreet_Sum09_Front_FNL.pdf 516533... (9 Replies)
Discussion started by: kenray
9 Replies
Login or Register to Ask a Question