Extract zip code information from address, put into new column


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract zip code information from address, put into new column
# 1  
Old 05-08-2009
Extract zip code information from address, put into new column

Hi, suppose I have a colon delimeterd file with address field like this

blue:john's hospital new haven CT 92881-2322
yellow:La times copr red road los angeles CA90381 1302
red:las vegas hotel sand drive Las vegas NV,21221

How do I create a new field that contain the zip code information only, extracted from the address field
The problem is with error tolerance, in the survey data file, there are many ways people wrote zip codes:


CT 92881-2322 (subzip code with hyphen)
CT 92881 2322 (subzip code with space)
CT 92881 (no subzip code)
CT,92881 (extra comma)
CT92881 (no space between state and zip code)



Do you think you can do this in python or awk? Thanks.
The new zip code field should be in standardized format:
CT 92881-2322 if there is sub zip code
CT 92881 if the sub zip code is empty
# 2  
Old 05-08-2009
Python
Code:
#!/usr/bin/env python

states=["CA","CT","NV"] #get all the states code here
for line in open("file"):
    line=line.strip()
    for s in states:        
        if s in line:
            print line[line.index(s):]

output:
Code:
 # ./test.py
CT 92881-2322
CA90381 1302
NV,21221

# 3  
Old 05-08-2009
Code:
awk '
{
 left = statezip = $0
 sub( / [A-Z][A-Z].*/,"",left)
 sub( left " ","", statezip)
 state=substr(statezip,1,2)
 sub( /[A-Z][A-Z][, ]*/,"", statezip)
 sub( " ", "-", statezip)
 printf "%s:%s %s\n", left, state, statezip
}' "$FILE"

In the shell, if the file is not too big:

Code:
while IFS= read -r line
do
  left=${line% [A-Z][A-Z]*}
  statezip=${line#"$left "}
  zip=${statezip#??}
  state=${statezip%"$zip"}
  state=${state%[, ]}
  zip=${zip#[ ,]}
  case $zip in
    *\ *) lzip=${zip% *}
          rzip=${zip#* }
          zip=$lzip-$rzip
          ;;
  esac
  printf "%s:%s %s\n" "$left" "$state" "$zip"
done < "$FILE"

# 4  
Old 05-10-2009
thanks a lot cfajohnson

But do you think sub( / [A-Z][A-Z].*/,"",left) is prone to unintentionally pick up patterns for example in street names, rather than in zip.

Is there a possible way to make it into sub(/ [two letter patern in the list.[0-9][0-9][0-9][0-9][0-9]/,"",left)

list="AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA,
HI, ID, IL, IN, IA, KS, KY, LA, ME, MD,
MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ,
NM, NY, NC, ND, OH, OK, OR, PA, RI, SC,
SD, TN, TX, UT, VT, VA, WA, WV, WI, WY"

thanks
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Extract .zip file without using unzip,tar

Hi, Need to extract a file containing multiple .txt files without using tar/unzip as they are not available (1 Reply)
Discussion started by: santoshdrkr
1 Replies

2. Shell Programming and Scripting

Curl download zip extract large xml file

Hi i have a php script that works 100% however i don't want this to run on php because of server limits etc. Ideally if i could convert this simple php script to a shell script i can set it up to run on a cron. My mac server has curl on it. So i am assuming i should be using this to download the... (3 Replies)
Discussion started by: timgolding
3 Replies

3. Shell Programming and Scripting

Extract specific text from variable and put it into array

Dear community, I have to do something too hard for me :rolleyes:. I hope you can help me. This is an output coming from Oracle query, stored in a file called query.out, there are many rows, but I read them, one by one, using while/read/done. Assuming each row is contained into $line variable... (8 Replies)
Discussion started by: Lord Spectre
8 Replies

4. Shell Programming and Scripting

how to write bash script that will automatically extract zip file

i'm trying to write a bash script that that will automatically extract zip files after the download. i writed this script #!/bin/bash wget -c https://github.com/RonGokhle/kernel-downloader/zipball/master CURRENDIR=/home/kernel-downloader cd $CURRENDIR rm $CURRENDIR/zipfiles 2>/dev/null ... (2 Replies)
Discussion started by: ron gokhle
2 Replies

5. Shell Programming and Scripting

Copy column string and put in different column

Hello Here is my input: SU3902 SU3902A NS29C (10) (00) Q1J1 0 SU3902 SU3902B VLR05 (20) (02) Q2H1 4 SU3902 SU3902C NBR22 (30) (06) Q3R5 8 SU3904 SU39047 NSV19 (11) (09) Q4k6 2 SU3904 SU39048 LB231 (12) (05) Q5k1 6 SU3904 SU39049 11VLT (13) (08) Q10C1 10 SU3904 SU3904A 25R05 (15) (06)... (3 Replies)
Discussion started by: pareshkp
3 Replies

6. UNIX for Dummies Questions & Answers

How to extract contents of ONLY one directory from ZIP folder.

I have a ZIP folder ( folder.zip or folder.jar ) which contains , lets say 3 directories under it ( directory1, directory2, directory3). How can i unzip the contents of directory2 ONLY ? I do not have to unzip the complete folder. TIA. (2 Replies)
Discussion started by: Sheel
2 Replies

7. Programming

extracting information from lines, put them into arrays

hi I need a little help writing this small perl script. I'm trying to extract the values from each line in a file and find the average for example cat school Highschool 100, 123, 135 Middleschool 41, 67, 54 Elementary 76, 315, 384 ./average.pl highschool: 119.3 middleschool: 54... (2 Replies)
Discussion started by: gengar
2 Replies

8. Shell Programming and Scripting

how to put it in a column

Simple question ..What is the easiest way to list my output in a column like so dddd dddd dddd dddd output dddd dddd dddd dddd Thanks (3 Replies)
Discussion started by: bombcan
3 Replies

9. Shell Programming and Scripting

read a part of information from txt and put into the script

Hi, I have a name.txt which is stored: APPLE ORANGE RED BLUE GREEN and my script is: $name=`cat name.txt for file_number in `ls 1 /appl/CH_DATA/archive/CACHE/CDBACKUP$name*.archived however, my script cannot read name.txt and put into my scrip line, I would like the output is to... (18 Replies)
Discussion started by: happyv
18 Replies

10. IP Networking

Can i put 2 ip address in one nic?

I have a SCO OpenServer 5.0.5 and my cuestion is can i have 2 ip address in one nic?, how?. Thanks. (1 Reply)
Discussion started by: dvl
1 Replies
Login or Register to Ask a Question