Shell Programming and Scripting

Hello Unix gurus,

I have a gzipped file where each line contains 2 street addresses in the US. What I want to do is get a count for each state that does not match.

What I have so far is:
$ gzcat matched_10_09.txt.gz |cut -c 106-107,184-185 | head -5
CTCT
CTNY
CTCT
CTFL
CTMA

This cuts the State fields out of each line. What I want to do is count the number of occurrences that each State has where it is not the same. So I want to pipe the output to a sed or awk process that eliminates, in the example above, CTCT (but it also has to eliminate NYNY, FLFL, etc). It will compare the first 2 characters to the last 2 characters and if there is a match, delete (or skip). Then I will pipe the output to 'wc' and get my desired result. In the above case, it would be 3.

I am always very grateful for assistance regarding my questions, but sometimes I feel like each answer I get is like getting a fish as opposed to learning how to fish; so to speak. So, if you could briefly mention how your solution works, I would be MOST grateful.

Thanks in advance.

try it with awk

Code:

gzcat matched_10_09.txt.gz |awk '{if(substr($0,106,2)!=substr($0,184,2)){print substr($0,106,2)substr($0,184,2)}}'

Code:

gzcat matched_10_09.txt.gz | awk '{x=substr($0,106,2);y=substr($0,184,2);if(x != y){a[x]++;a[y]++}}END{for(i in a)print i,a[i]}'

That was very elegant! It actually gives a count for each state. I will try to parse the syntax so I can build such awk commands myself. Thanks.

Code:

awk 'BEGIN {FS=""} !($106$107==$184$185)' urfile

Code:

$ gzcat matched_10_09.txt.gz |sed '/.\{105\}\(..\).\{76\}\1/d'

-or-

Code:

$ gzcat matched_10_09.txt.gz |sed -r '/.{105}(..).{76}\1/d'