Shell Programming and Scripting

Ok, after pulling my hair out on this one, i need to ask for some help

I have a file called "final" that looks like the following (this is a section of a bigger file)

Code:

18-5-252906,180105,00041.00,42070461139120,AUTH CODE:630911,000,101
18-5-255398,170105,01594.00,42110524389101,AUTH CODE:066811,003,101
19-5-239531,170105,00628.00,42124415729001,AUTH CODE:636236,003,101
15-5-268103,170105,00395.13,42123345209101,AUTH CODE:083378,003,101
18-5-255630,170105,02108.00,42135735559001,AUTH CODE:3698  ,003,101
20-5-258607,180105,00015.88,42121026959101,AUTH CODE:014001,000,101
18-5-255906,180105,00040.00,42070461139100,AUTH CODE:630911,000,101
20-5-258611,180105,00040.25, 4205843099910,AUTH CODE:082704,000,101
20-5-258751,180105,00020.00,42103493309100,AUTH CODE:6850  ,001,101

----------------
My script looks like this

Code:

awk 'BEGIN { FS="," } { if ( $5$6 ~ /AUTH CODE.*(000|001|100)/ ) { print $0 } }' final > finaltemp
awk -F',' ' length(sprintf("%.0f",0+$4))!=14{print > "exceptions2";next} {print} ' finaltemp > WSECTRAD
awk 'BEGIN { FS="," } { if ( $6 ~ /055|002|003/ ) { print $0 } }' final > exceptions3
awk 'BEGIN { FS="," } { if ( $5 ~ /REFUND|REFERRAL/ ) { print $0 } }' final > exceptions1

the script process
------------------

As you can see from my script above, I run some awk commands on this file called "final" to extract the data i need.. there are 7 columns in total in the file seperated by a comma.

1) My first line of awk looks at the 5th and 6th column for the words AUTH CODE and a three letter code 000, 100, or 001 respectivly and then extracts to a file called finaltemp

2) The next line, checks to see if the 4th column is indeed 14 characters, if not then chucks it to an exceptions file , it then creates a file called WSECTRAD with all records that ok

3) The next awk extracts column 6 values of 055, 002 and 003 to another exceptions file

4) And finally another awk looking at column 5 for some keywords, again throwing them out to an exceptions file



Now that all seesm to work great, except strangely on a couple of occasions where a single 003 (in the 6th column) seems to make its way into the WSECTRAD file AND the exceptions file ?!? ....I can obviously understand why it goes to exceptions, because a 003 is an exception.....but why does this one rogue entry end up in WSECTRAD ??

here is the culprit record for today...see below



18-5-255398,170105,01594.00,42110524389101,AUTH CODE:066811,003,102




As you can see from the section of the "final" file at the top,(its the second one from the top) it is not the only 003 at all, infact there are loads of these throughout the file, and unless I am going blind, I cant see the difference between this record and any of the other 003's (as far as the awk scripts are concerned anyway)

Can anybody spot any reasoning why this one record is the only 003 to hit the WSECTRAD file ??? (as mentioned before, A copy does also go to the correct exceptions file aswell)


Any help on this would be greatly appreciated

I changed

$5$6 ~ /AUTH CODE.*(000|001|100)/

to

$5$6 ~ /AUTH CODE.*(000|001|100)$/

(notice the dollar to explicity define the end of the 6th field) in the first awk and that seems to fix it for me.

Cheers
ZB

ill give it a go, thankyou

The reason it was bringing in the record with "003" in it was because the end of the 5th field for that record ended in "1". When you concatenate the two fields with $5$6, you end up with "1003" at the end of $5$6 which matches the rule you defined in the first awk. Therefore, you need the "$" at the end of the regex to tell it that the (000|001|100) is anchored at the very end of the fields in the match.

Cheers
ZB