Shell Programming and Scripting

First, I am trying to search for certain string types within a very large file. Here is a sample record:

Code:

MyCountry|MyGroup|MyCust|Run-Date|N/A|SERVER|OS|USERID|897//123456//LNAME FNAME|STATE|LLOGON

In field 9 I am looking for invalid formats. A valid format would be 897/C/123456//LNAME FNAME. The leading 897 can be numeric or it can be character (897 or US, these are country codes).

I am trying to convert a grep regexp search into an awk search with little success. The reason being is that I have to read in the line with grep, then test the variable, then print out the whole line. I figured with awk I could be doing the whole thing with one line and I could get rid of a slow "while read LINE;do" statement which makes my script extremely slow. Below are my two line examples:

GREP:

Code:

cat SID-Removed.csv | cut -d"|" -f9 |grep -ih '[[:alnum:]]\{2,3\}"//"[[:alnum:]]\{6\}'   (this one finally worked)

Note: I had tried

Code:

grep -ih '[[:alnum:]]\{2,3\}//[[:alnum:]]\{6\}'

but it did not produce the desired results

AWK:

Code:

awk -F'|' 'BEGIN { search_regex = "[[:alnum:]]\{2,3\}//[[:alnum:]]\{6\}" }tolower($9) ~ search_regex  {print $0}' SID-Removed.csv

I even tried this:

Code:

awk -F'|' 'BEGIN { search_regex = "[[:alnum:]]\{2,3\}\/\/[[:alnum:]]\{6\}"  }tolower($9) ~ search_regex  {print $0}' SID-Removed.csv

The awk statement finds 897/C/123456/LNAME FNAME, but it does not find the ones where the second element of that string is blank (897//123456//LNAME FNAME). Can anyone help me figure out what i'm doing wrong?

Moderator's Comments:

Video tutorial on how to use code tags in The UNIX and Linux Forums.

I'm not sure [[:alnum:]] is supported in awk. Try [0-9a-zA-Z].

I don't think you need to escape the { } with \.

alnum is POSIX character class. gawk is ok with those. the { } is what is not supported by many awk. it is newer i guess and gawk didn't want to break itself

Code:

[mute@geek ~]$ echo foo | gawk '/^[[:alnum:]]{3}$/'
[mute@geek ~]$ echo foo | gawk --posix '/^[[:alnum:]]{3}$/'
foo

so to help more, please tell me which awk and/or OS you use.

Code:

awk -F\| '$9 ~ "^[a-zA-Z0-9]{2,3}/[a-zA-Z0-9]?/[a-zA-Z0-9]{6}"' file

I was trying to use [[:alnum:]]\{2,3\} because I want to search for any alpha (0-9, a-z as you indicated), but only if it found 2 or 3 before the /

897//123456
US//123456

If I switch to [0-9a-zA-z], can I still use the count feature from grep where I tell it to only match 2 or 3 characters in that position?

Yes you can...

Here is a full code snippet of what i'm trying to convert. Note that I am currently using grep in a while look and reading through a file with millions of records make this take quite a long time to complete:

Code:

while read LINE
do
   REC13=`echo $LINE |cut -d"|" -f9 |grep -ih '[[:alnum:]]\{2,3\}"//"[[:alnum:]]\{6\}'`
   if [ -n "$REC13" ]
   then
        echo $LINE >> ./$PRVYR/$MONTH/mislabeled/$MONTH-mislabeled.csv
   fi
done < INFILE

This particular record looks for these strings: CCC//SSSSSS or CC//SSSSSS

My goal is to try and convert this into an awk command.

Moderator's Comments:

Code tags for code, please.