Shell Programming and Scripting

Hi,

I have the following requirement.

I have one lookup file which contains 15 columns and 7000 records.

Ex:

123,MEDICA,134,145,1178,123,678,345,2345,HP,COL,K12,SR,OX,78919

I have input file which contains 14 columns and 20 million records.Some times the record count is more than 20 million.

I need to match the first 14 columns with 14 columns in the lookup file and if matches i need the 15 the column.

Can any one please help me and give me some code which will process very effeciently.
I need to finish this process less than 20 minutes.

Can any one please suggest me how can we acheive this using ksh or even perl script is also good.

Thanks

Code:

#!/usr/bin/perl -w
$lookupfile=$ARGV[0];
open(LOOKUP,"<$lookupfile") || die "Argh: $lookupfile : $!";
while (<LOOKUP>) {
    if (/(.*)\,([^\,])$/) {
        $lookup{$1}=$2;
    }
}
close (LOOKUP);

while(<STDIN>) {
    $key=$_;
    chomp $key;
    if ($match=$lookup{$key}) {
        print "${key},${match}\n";
    } else {
        print "${key},ERROR!\n";
    }
}

Usage: scriptname.pl lookupfile.txt < inputfile.txt

Thanks for your reply.And there is small change in the requirement.
Lookupfile

123,MED,134,145,(117-200),<ANY>,678,345,234,HP,CO,K12,SR,OX,7891

InPutFile

123,MEDICA,134,145,118,123,678,345,2345,HP,COL,K12,SR,OX

Here column highlited in red in the input text file which falls in between 117-200(range comparision) and column highlighted in rose colour in the input file equals <ANY> in the lookupfile.Can you please tell me how to acheive this by using the above code.

I used the above script and i am getting the following output

#!/usr/bin/perl -w
$lookupfile=$ARGV[0];
open(LOOKUP,"<$lookupfile") || die "Argh: $lookupfile : $!";
while (<LOOKUP>) {
if (/(.*)\,([^\,])$/) {
$lookup{$1}=$2;
}
}
close (LOOKUP);
while(<STDIN>) {
$key=$_;
chomp $key;
if ($match=$lookup{$key}) {
print "${key},${match}\n";
} else {
print "${key},ERROR!\n";
}
}


scripname.pl lkpfile.txt < inputfile.txt
123,123,123,123,123,123,123,123,123,123,123,123,123,123,ERROR!
cat lkpfile.txt
123,123,123,123,123,123,123,123,123,123,123,123,123,123,uma
cat inputfile.txt
123,123,123,123,123,123,123,123,123,123,123,123,123,123

I think you've got those files the wrong way around. I would expect inputfile.txt to be the input data (i.e. with "uma") and lkpfile.txt to be the keys you are looking up.