Shell Programming and Scripting

Hi,
I have a task where i need to compare columns of two files.

First file is $REG_InputFileName:

"UPDATE","1010021126","1-01-01","USD"
"UPDATE","1013000101","1-01-01","THB"
"UPDATE","1013010107","1-01-01","THB"
"UPDATE","1110011122","1-01-01","USD"
"UPDATE","1110020111","1-01-01","THB"
"UPDATE","1110020111","1-01-01","USD"
"UPDATE","1110020113","1-01-01","THB"
"UPDATE","1620020409","1-01-01","USD"
"UPDATE","1620020612","1-01-01","USD"
"UPDATE","1620021126","1-01-01","USD"
"UPDATE","1910020411","1-01-01","USD"
"UPDATE","1910020722","1-01-01","USD"
"UPDATE","1910030021","1-01-01","THB"
"UPDATE","1910031191","1-01-01","THB"
"UPDATE","1910031221","1-01-01","THB"
"UPDATE","2620021162","1-01-01","USD"


Second file is $TargetSeqPath/Ref.tmp:

001.,ARE
002.,ARE
003.,ARE
011.,ARE
012.,ARE
021.,BHR
031.,BHR
041.,QAT
043.,QAT
051.,QAT
061.,OMN
071.,OMN
081.,LBN
091.,LBN
101.,JOR
111.,JOR
121.,EGY
131.,EGY
141.,MUS
151.,MUS
162.,IRQ
181.,PAK
191.,PAK


In this, i need to compare first three characters of 2nd column of $REG_InputFileName with first 3 characters 1st column of $TargetSeqPath/Ref.tmp

if there is match then the whole record should go to a file which is in the directory named same as second column of $TargetSeqPath/Ref.tmp

for e.g.

first record of $REG_InputFileName is

"UPDATE","1010021126","1-01-01","USD"
so first three characters will be 101 now i need to lookup this 101 into $TargetSeqPath/Ref.tmp and there is match ;
001.,ARE

then this whole record "UPDATE","1010021126","1-01-01","USD" should go to a directory named as ARE

i have following code for this :

while read i
do
echo $i > $TargetSeqPath/Ref.tmp
OutDir=`awk -F"," '{print $2}' $TargetSeqPath/Ref.tmp`
awk -F"," 'NR==FNR {col1[substr($1,1,3)]=$2} NR!=FNR { if (col1[substr($2,2,3)]=="") {next} print $0}' $TargetSeqPath/Ref.tmp $REG_InputFileName >>$Target
SeqPath/$OutDir/$OutFile-$mdt.txt
done < $RefInputFileName


this is code is working fine and i am getting everything mentioned above. But the problem is if there is match.
for e.g.

"UPDATE","2620021162","1-01-01","USD"

then this record should go to a directory named as "others"

Can anybody help me how to check this mismatch and send the record to other directory

Thanks

Assuming only one record per type (except for others).

Use nawk or /usr/xpg4/bin/awk on Solaris.
You may need to adjust the names of the files.

Code:

awk 'NR == FNR { ref[$1] = $2; next }
{ 
  if (substr($2, 1, 3) in ref) { 
    dest = ref[substr($2, 1, 3)] "/"
    system("[ -d " dest " ] || mkdir " dest) 
    print > (dest FILENAME)
    close(dest FILENAME)
    }
  else {
    dest = "others/"
    system("[ -d " dest " ] || mkdir " dest)
    print > (dest FILENAME)
    }
}' FS='[.],' Ref.tmp FS='","' REG_InputFileName

Hi,
Thanx for helping me out...
actually i am new at scripting ...can you explain me a liitle bit what your code is doing?? so that if at sometime i need to make changes in it...i can make..

Yes,
while reading the first input file build an associative array ref keyed by the first field with values from the second one. You should add more control here: I'm assuming that the second field is always present and that it does not contain special (for the shell) characters.
While reading the second input file check if the first 3 characters match any ref key, if yes: checks if a directory with that name already exists, if not creates it. Then prints the current record in a file named as the second input filename in that directory; if not: does the same for the others directory.

Hope this helps.

Hi Radoulov,
Thanks for the explanation. i understand it now....but your code is not working ..it's giving me error while fetching every record:

OK,
could you run the following commands (see example below) with your files and post the entire terminal output?
You should preserve the formatting!

1.

Code:

ls -l

2.

Code:

awk 'NR == FNR { ref[$1] = $2; next }
{ 
  if (substr($2, 1, 3) in ref) { 
    dest = ref[substr($2, 1, 3)] "/"
    system("[ -d " dest " ] || mkdir " dest) 
    print > (dest FILENAME)
    close(dest FILENAME)
    }
  else {
    dest = "others/"
    system("[ -d " dest " ] || mkdir " dest)
    print > (dest FILENAME)
    }
}' FS='[.],' Ref.tmp FS='","' REG_InputFileName

3.

Code:

ls -lR

Something like this:

Code:

% ls -l
total 8
-rw-r--r-- 1 radoulov radoulov 207 2008-11-07 23:09 Ref.tmp
-rw-r--r-- 1 radoulov radoulov 608 2008-11-07 23:08 REG_InputFileName
% 
% 
% awk 'NR == FNR { ref[$1] = $2; next }
{
  if (substr($2, 1, 3) in ref) {
    dest = ref[substr($2, 1, 3)] "/"
    system("[ -d " dest " ] || mkdir " dest)
    print > (dest FILENAME)
    close(dest FILENAME)
    }
  else {
    dest = "others/"
    system("[ -d " dest " ] || mkdir " dest)
    print > (dest FILENAME)
    }
}' FS='[.],' Ref.tmp FS='","' REG_InputFileName
% ls -lR
.:
total 24
drwxr-xr-x 2 radoulov radoulov 4096 2008-11-07 23:09 IRQ
drwxr-xr-x 2 radoulov radoulov 4096 2008-11-07 23:09 JOR
drwxr-xr-x 2 radoulov radoulov 4096 2008-11-07 23:09 others
drwxr-xr-x 2 radoulov radoulov 4096 2008-11-07 23:09 PAK
-rw-r--r-- 1 radoulov radoulov  207 2008-11-07 23:09 Ref.tmp
-rw-r--r-- 1 radoulov radoulov  608 2008-11-07 23:08 REG_InputFileName

./IRQ:
total 4
-rw-r--r-- 1 radoulov radoulov 38 2008-11-07 23:09 REG_InputFileName

./JOR:
total 4
-rw-r--r-- 1 radoulov radoulov 38 2008-11-07 23:09 REG_InputFileName

./others:
total 4
-rw-r--r-- 1 radoulov radoulov 38 2008-11-07 23:09 REG_InputFileName

./PAK:
total 4
-rw-r--r-- 1 radoulov radoulov 38 2008-11-07 23:09 REG_InputFileName
%