Shell Programming and Scripting

Hi Folks,

I need to compare two very huge file ( i.e the files would contain a minimum of 70k records each) using awk or sed. The comparison needs to be done with respect to a 'key'. For example :

File1
**********
1234|TONY|Y75634|20/07/2008
1235|TINA|XCVB56|30/07/2009
43456|PATS|U74454|12/04/2009
23456|DAPS|R4576|15/03/2008

File2
******
1235|TINA|XCVB56|30/07/2009
1234|TONY|Y75634|20/07/2008
23456|DAPS|R4576|15/03/2008

In this case, if I consider '|' as the delimiter , the value at column 3 as 'key' for the files, I need to look out for this key in the second file and once that is got, I need to compare the values at column 2 of the corresponding records in both the files.

Also, I need to report a message in case the key is not present in file2.

PS: I have a perl script running for this.. but it takes way too long to perform this comparison, your help in suggesting some awk script which would perform this action much faster would be really appreciated.

Thanks in advance
Ranjani

What should be the desired output?

Sorry, i had forgotten to mention the desired output.

The output needs to be logged in file3 which says either : " the correspoding records matched" or " the corresponding records did not match" or "the key in file1 does not exist in file2"

Thanks
Ranjani

Try something like this:

Code:

awk -F"|" '
NR==FNR{a[$3]=$2;next}
a[$3]==$2{print $0 " <= Corresponding records match";next}
a[$3]{print $0 " <= "Corresponding records did not match"}
{print $0 " <= Key File1 not exist in File2"}
' File1 File2

Regards

Another one:
(use nawk or /usr/xpg4/bin/awk on Solaris)

Code:

awk>file3 -F\| 'NR==FNR{f1[$3]=$2;next}
{print "key",$3 in f1?$3" records "(f1[$3]==$2?"":"do not ")\
 "match":$3" is missing"}' file2 file1

Hi Franklin and Radoulov.. thanks a lot for your responseS, but I am getting the following o/p when i run this script :
******************************************************
1234 RANJ 45678 y786 <= Corresponding records did not match
1234 RANJ 45678 y786 <= Key File1 not exist in File2
567 SREE 3457 xg456 <= Corresponding records did not match
567 SREE 3457 xg456 <= Key File1 not exist in File2
34567 TANGO 4567 H7694 <= Corresponding records did not match
34567 TANGO 4567 H7694 <= Key File1 not exist in File2
4567 qrea 3456 but731 <= Corresponding records did not match
4567 qrea 3456 but731 <= Key File1 not exist in File2
34567 TANGO 4567 H7694 <= Corresponding records did not match
34567 TANGO 4567 H7694 <= Key File1 not exist in File2
567 SREE 3457 xg456 <= Corresponding records did not match
567 SREE 3457 xg456 <= Key File1 not exist in File2
1234 RANJ 45678 y786 <= Corresponding records did not match
1234 RANJ 45678 y786 <= Key File1 not exist in File2

*******************************************************

This is not the desired o/p. Please could you help :

The i/p files are :
************************
1234 RANJ 45678 y786
567 SREE 3457 xg456
34567 TANGO 4567 H7694
4567 qrea 3456 but731

*****************************
File2:
******************
34567 XAXRO 4567 H7694
567 SREE 3457 xg456
1234 RANJ 45678 y786

The desired output file is :


For the key:45678 , the fields are matching
For the key:3457, the fields are matching
For the key : 4567 , the fileds are not matching
For the key : 45678 , record is not present in file2.

*******************************************

Please provide your useful inputs. Your help would be much appreciated.

Thanks a lot

Ranjani

Perhaps I'm missing something,
this is what I get from the code I posted:
(you changed the field separator so the -F switch is removed)

Code:

$ head file[12]
==> file1 <==
1234 RANJ 45678 y786
567 SREE 3457 xg456
34567 TANGO 4567 H7694
4567 qrea 3456 but731

==> file2 <==
34567 XAXRO 4567 H7694
567 SREE 3457 xg456
1234 RANJ 45678 y786
$ nawk  'NR==FNR{f1[$3]=$2;next}
{print "key",$3 in f1?$3" records "(f1[$3]==$2?"":"do not ")\
 "match":$3" is missing"}' file2 file1
key 45678 records match
key 3457 records match
key 4567 records do not match
key 3456 is missing

You say you want to mark the key 45678 as not present, but it is ...