A small project with bash script and craigslist


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting A small project with bash script and craigslist
# 1  
Old 11-25-2008
Lightbulb A small project with bash script and craigslist

Before I start, I'm posting this for community knowledge, not to offend anyone. I thought it was an interesting project so that's why I'm sharing it.

In the notorious erotic classified ads of craigslist, thousands of individuals post classifieds, using the internet to sell themselves with an ease never before seen in history. Many resort to posting their phone numbers for their prospective clientele and establish a time and location where the exchange of money and sex will begin.

As these individuals move from place to place, they post their contact information in the respective city/state within craigslist in order to target the appropriate audience. These individuals will maintain their phone number as they move from city to city. This then leaves a record of a phone, a location and a time.

Using this data (phone numbers posted, time posted and location of the post), I've created a bash shell script that generates a Google Earth KML file, depicting the movements of these phone numbers as they post to the craigslist erotic services page relative to the city they are in.

This script:
1) Downloads the latest RSS feeds from all the erotic classifieds in all the craigslist websites relative to its city/state.
2) Obtains the post page URL where individuals post their information and downloads it into a directory named after its respective city/state.
3) Dumps in raw text the html output (from links text browser) into a "raw data" file, modifies this data, extracting and saving the list of phone numbers per city/state into a file and naming this file with the date and time.
4) Uses awk to create a table based on the phone number, date saved and location saved on craigslist.
5) Adds a decimal latitude and longitude for the city where the classified was psoted, making the table have four fields: phone, date, location name and location latlongs in decimal form.
6) Finally, using awk again, formats this data into a KML file so it can be read by Google Earth. The file will be dumped in the "dbout" directory as "dbout.kml."

A couple of notes: Sorry, I don't create anything outside this platform but I'm sure you could adjust it to fit your needs in whatever OS you run. Also, I'm self taught so if you notice there's a better way of doing something, don't bash. We're all here to learn.

This was a little project that I figured would be interesting to do so I chose the erotic ads on craigslist because it was simply a reliable data set. Plus, it would be interesting to see how far the industry goes to do what it does.

In some cases, I've noticed false positives. The script looks for a 10-digit number in the posts so some data may be a little jacked.

I recommend running this within a directory and not in your /home/user directory or something. You need wget, cat, rpl, awk, links, sort, grep, uniq, column and find to run it so make sure you get them before you do. Save the script as begin.sh and execute it with "sh begin.sh".

Anyways, here's the script.
------------------------------------
Code:
clear
rm -rf cities/*
rm -rf data/* 
rm -rf dbout/tmp
mkdir dbout/tmp
echo "Downloading RSS feeds."
wget -c -nc -q -t 0 --user-agent="" "http://abilene.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/abilene/
wget -c -nc -q -t 0 --user-agent="" "http://akroncanton.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/akroncanton/
wget -c -nc -q -t 0 --user-agent="" "http://albany.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/albany/
wget -c -nc -q -t 0 --user-agent="" "http://albuquerque.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/albuquerque/
wget -c -nc -q -t 0 --user-agent="" "http://allentown.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/allentown/
wget -c -nc -q -t 0 --user-agent="" "http://altoona.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/altoona/
wget -c -nc -q -t 0 --user-agent="" "http://amarillo.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/amarillo/
wget -c -nc -q -t 0 --user-agent="" "http://ames.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/ames/
wget -c -nc -q -t 0 --user-agent="" "http://anchorage.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/anchorage/
wget -c -nc -q -t 0 --user-agent="" "http://annapolis.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/annapolis/
wget -c -nc -q -t 0 --user-agent="" "http://annarbor.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/annarbor/
wget -c -nc -q -t 0 --user-agent="" "http://appleton.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/appleton/
wget -c -nc -q -t 0 --user-agent="" "http://asheville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/asheville/
wget -c -nc -q -t 0 --user-agent="" "http://athensga.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/athensga/
wget -c -nc -q -t 0 --user-agent="" "http://athensohio.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/athensohio/
wget -c -nc -q -t 0 --user-agent="" "http://atlanta.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/atlanta/
wget -c -nc -q -t 0 --user-agent="" "http://auburn.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/auburn/
wget -c -nc -q -t 0 --user-agent="" "http://augusta.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/augusta/
wget -c -nc -q -t 0 --user-agent="" "http://austin.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/austin/
wget -c -nc -q -t 0 --user-agent="" "http://bakersfield.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/bakersfield/
wget -c -nc -q -t 0 --user-agent="" "http://baltimore.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/baltimore/
wget -c -nc -q -t 0 --user-agent="" "http://batonrouge.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/batonrouge/
wget -c -nc -q -t 0 --user-agent="" "http://beaumont.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/beaumont/
wget -c -nc -q -t 0 --user-agent="" "http://bellingham.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/bellingham/
wget -c -nc -q -t 0 --user-agent="" "http://bend.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/bend/
wget -c -nc -q -t 0 --user-agent="" "http://bgky.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/bgky/
wget -c -nc -q -t 0 --user-agent="" "http://bham.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/bham/
wget -c -nc -q -t 0 --user-agent="" "http://binghamton.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/binghamton/
wget -c -nc -q -t 0 --user-agent="" "http://blacksburg.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/blacksburg/
wget -c -nc -q -t 0 --user-agent="" "http://bloomington.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/bloomington/
wget -c -nc -q -t 0 --user-agent="" "http://bn.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/bn/
wget -c -nc -q -t 0 --user-agent="" "http://boise.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/boise/
wget -c -nc -q -t 0 --user-agent="" "http://boone.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/boone/
wget -c -nc -q -t 0 --user-agent="" "http://boston.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/boston/
wget -c -nc -q -t 0 --user-agent="" "http://boulder.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/boulder/
wget -c -nc -q -t 0 --user-agent="" "http://brownsville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/brownsville/
wget -c -nc -q -t 0 --user-agent="" "http://brunswick.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/brunswick/
wget -c -nc -q -t 0 --user-agent="" "http://buffalo.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/buffalo/
wget -c -nc -q -t 0 --user-agent="" "http://burlington.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/burlington/
wget -c -nc -q -t 0 --user-agent="" "http://capecod.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/capecod/
wget -c -nc -q -t 0 --user-agent="" "http://carbondale.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/carbondale/
wget -c -nc -q -t 0 --user-agent="" "http://catskills.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/catskills/
wget -c -nc -q -t 0 --user-agent="" "http://cedarrapids.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/cedarrapids/
wget -c -nc -q -t 0 --user-agent="" "http://centralmich.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/centralmich/
wget -c -nc -q -t 0 --user-agent="" "http://chambana.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/chambana/
wget -c -nc -q -t 0 --user-agent="" "http://charleston.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/charleston/
wget -c -nc -q -t 0 --user-agent="" "http://charlestonwv.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/charlestonwv/
wget -c -nc -q -t 0 --user-agent="" "http://charlotte.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/charlotte/
wget -c -nc -q -t 0 --user-agent="" "http://charlottesville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/charlottesville/
wget -c -nc -q -t 0 --user-agent="" "http://chattanooga.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/chattanooga/
wget -c -nc -q -t 0 --user-agent="" "http://chautauqua.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/chautauqua/
wget -c -nc -q -t 0 --user-agent="" "http://chicago.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/chicago/
wget -c -nc -q -t 0 --user-agent="" "http://chico.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/chico/
wget -c -nc -q -t 0 --user-agent="" "http://cincinnati.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/cincinnati/
wget -c -nc -q -t 0 --user-agent="" "http://clarksville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/clarksville/
wget -c -nc -q -t 0 --user-agent="" "http://cleveland.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/cleveland/
wget -c -nc -q -t 0 --user-agent="" "http://cnj.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/cnj/
wget -c -nc -q -t 0 --user-agent="" "http://collegestation.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/collegestation/
wget -c -nc -q -t 0 --user-agent="" "http://columbia.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/columbia/
wget -c -nc -q -t 0 --user-agent="" "http://columbiamo.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/columbiamo/
wget -c -nc -q -t 0 --user-agent="" "http://columbus.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/columbus/
wget -c -nc -q -t 0 --user-agent="" "http://columbusga.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/columbusga/
wget -c -nc -q -t 0 --user-agent="" "http://corpuschristi.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/corpuschristi/
wget -c -nc -q -t 0 --user-agent="" "http://corvallis.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/corvallis/
wget -c -nc -q -t 0 --user-agent="" "http://cosprings.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/cosprings/
wget -c -nc -q -t 0 --user-agent="" "http://dallas.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/dallas/
wget -c -nc -q -t 0 --user-agent="" "http://danville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/danville/
wget -c -nc -q -t 0 --user-agent="" "http://daytona.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/daytona/
wget -c -nc -q -t 0 --user-agent="" "http://dayton.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/dayton/
wget -c -nc -q -t 0 --user-agent="" "http://decatur.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/decatur/
wget -c -nc -q -t 0 --user-agent="" "http://delaware.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/delaware/
wget -c -nc -q -t 0 --user-agent="" "http://denver.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/denver/
wget -c -nc -q -t 0 --user-agent="" "http://desmoines.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/desmoines/
wget -c -nc -q -t 0 --user-agent="" "http://detroit.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/detroit/
wget -c -nc -q -t 0 --user-agent="" "http://dothan.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/dothan/
wget -c -nc -q -t 0 --user-agent="" "http://dubuque.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/dubuque/
wget -c -nc -q -t 0 --user-agent="" "http://duluth.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/duluth/
wget -c -nc -q -t 0 --user-agent="" "http://easternshore.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/easternshore/
wget -c -nc -q -t 0 --user-agent="" "http://eastidaho.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/eastidaho/
wget -c -nc -q -t 0 --user-agent="" "http://eastnc.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/eastnc/
wget -c -nc -q -t 0 --user-agent="" "http://eastoregon.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/eastoregon/
wget -c -nc -q -t 0 --user-agent="" "http://easttexas.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/easttexas/
wget -c -nc -q -t 0 --user-agent="" "http://eauclaire.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/eauclaire/
wget -c -nc -q -t 0 --user-agent="" "http://elmira.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/elmira/
wget -c -nc -q -t 0 --user-agent="" "http://elpaso.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/elpaso/
wget -c -nc -q -t 0 --user-agent="" "http://erie.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/erie/
wget -c -nc -q -t 0 --user-agent="" "http://eugene.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/eugene/
wget -c -nc -q -t 0 --user-agent="" "http://evansville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/evansville/
wget -c -nc -q -t 0 --user-agent="" "http://fargo.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/fargo/
wget -c -nc -q -t 0 --user-agent="" "http://farmington.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/farmington/
wget -c -nc -q -t 0 --user-agent="" "http://fayar.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/fayar/
wget -c -nc -q -t 0 --user-agent="" "http://fayetteville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/fayetteville/
wget -c -nc -q -t 0 --user-agent="" "http://flagstaff.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/flagstaff/
wget -c -nc -q -t 0 --user-agent="" "http://flint.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/flint/
wget -c -nc -q -t 0 --user-agent="" "http://florencesc.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/florencesc/
wget -c -nc -q -t 0 --user-agent="" "http://fortcollins.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/fortcollins/
wget -c -nc -q -t 0 --user-agent="" "http://fortmyers.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/fortmyers/
wget -c -nc -q -t 0 --user-agent="" "http://fortsmith.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/fortsmith/
wget -c -nc -q -t 0 --user-agent="" "http://fortwayne.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/fortwayne/
wget -c -nc -q -t 0 --user-agent="" "http://fredericksburg.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/fredericksburg/
wget -c -nc -q -t 0 --user-agent="" "http://fresno.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/fresno/
wget -c -nc -q -t 0 --user-agent="" "http://gadsden.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/gadsden/
wget -c -nc -q -t 0 --user-agent="" "http://gainesville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/gainesville/
wget -c -nc -q -t 0 --user-agent="" "http://galveston.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/galveston/
wget -c -nc -q -t 0 --user-agent="" "http://goldcountry.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/goldcountry/
wget -c -nc -q -t 0 --user-agent="" "http://grandisland.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/grandisland/
wget -c -nc -q -t 0 --user-agent="" "http://grandrapids.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/grandrapids/
wget -c -nc -q -t 0 --user-agent="" "http://greenbay.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/greenbay/
wget -c -nc -q -t 0 --user-agent="" "http://greensboro.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/greensboro/
wget -c -nc -q -t 0 --user-agent="" "http://greenville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/greenville/
wget -c -nc -q -t 0 --user-agent="" "http://gulfport.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/gulfport/
wget -c -nc -q -t 0 --user-agent="" "http://harrisburg.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/harrisburg/
wget -c -nc -q -t 0 --user-agent="" "http://harrisonburg.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/harrisonburg/
wget -c -nc -q -t 0 --user-agent="" "http://hartford.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/hartford/
wget -c -nc -q -t 0 --user-agent="" "http://hattiesburg.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/hattiesburg/
wget -c -nc -q -t 0 --user-agent="" "http://hickory.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/hickory/
wget -c -nc -q -t 0 --user-agent="" "http://hiltonhead.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/hiltonhead/
wget -c -nc -q -t 0 --user-agent="" "http://honolulu.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/honolulu/
wget -c -nc -q -t 0 --user-agent="" "http://houston.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/houston/
wget -c -nc -q -t 0 --user-agent="" "http://hudsonvalley.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/hudsonvalley/
wget -c -nc -q -t 0 --user-agent="" "http://humboldt.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/humboldt/
wget -c -nc -q -t 0 --user-agent="" "http://huntington.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/huntington/
wget -c -nc -q -t 0 --user-agent="" "http://huntsville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/huntsville/
wget -c -nc -q -t 0 --user-agent="" "http://imperial.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/imperial/
wget -c -nc -q -t 0 --user-agent="" "http://indianapolis.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/indianapolis/
wget -c -nc -q -t 0 --user-agent="" "http://inlandempire.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/inlandempire/
wget -c -nc -q -t 0 --user-agent="" "http://iowacity.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/iowacity/
wget -c -nc -q -t 0 --user-agent="" "http://ithaca.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/ithaca/
wget -c -nc -q -t 0 --user-agent="" "http://jackson.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/jackson/
wget -c -nc -q -t 0 --user-agent="" "http://jacksontn.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/jacksontn/
wget -c -nc -q -t 0 --user-agent="" "http://jacksonville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/jacksonville/
wget -c -nc -q -t 0 --user-agent="" "http://janesville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/janesville/
wget -c -nc -q -t 0 --user-agent="" "http://jerseyshore.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/jerseyshore/
wget -c -nc -q -t 0 --user-agent="" "http://jonesboro.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/jonesboro/
wget -c -nc -q -t 0 --user-agent="" "http://joplin.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/joplin/
wget -c -nc -q -t 0 --user-agent="" "http://jxn.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/jxn/
wget -c -nc -q -t 0 --user-agent="" "http://kalamazoo.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/kalamazoo/
wget -c -nc -q -t 0 --user-agent="" "http://kansascity.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/kansascity/
wget -c -nc -q -t 0 --user-agent="" "http://keys.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/keys/
wget -c -nc -q -t 0 --user-agent="" "http://killeen.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/killeen/
wget -c -nc -q -t 0 --user-agent="" "http://knoxville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/knoxville/
wget -c -nc -q -t 0 --user-agent="" "http://kpr.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/kpr/
wget -c -nc -q -t 0 --user-agent="" "http://ksu.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/ksu/
wget -c -nc -q -t 0 --user-agent="" "http://lacrosse.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/lacrosse/
wget -c -nc -q -t 0 --user-agent="" "http://lafayette.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/lafayette/
wget -c -nc -q -t 0 --user-agent="" "http://lakecharles.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/lakecharles/
wget -c -nc -q -t 0 --user-agent="" "http://lakeland.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/lakeland/
wget -c -nc -q -t 0 --user-agent="" "http://lancaster.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/lancaster/
wget -c -nc -q -t 0 --user-agent="" "http://lansing.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/lansing/
wget -c -nc -q -t 0 --user-agent="" "http://laredo.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/laredo/
wget -c -nc -q -t 0 --user-agent="" "http://lascruces.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/lascruces/
wget -c -nc -q -t 0 --user-agent="" "http://lasvegas.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/lasvegas/
wget -c -nc -q -t 0 --user-agent="" "http://lawrence.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/lawrence/
wget -c -nc -q -t 0 --user-agent="" "http://lawton.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/lawton/
wget -c -nc -q -t 0 --user-agent="" "http://lexington.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/lexington/
wget -c -nc -q -t 0 --user-agent="" "http://limaohio.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/limaohio/
wget -c -nc -q -t 0 --user-agent="" "http://lincoln.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/lincoln/
wget -c -nc -q -t 0 --user-agent="" "http://littlerock.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/littlerock/
wget -c -nc -q -t 0 --user-agent="" "http://logan.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/logan/
wget -c -nc -q -t 0 --user-agent="" "http://longisland.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/longisland/
wget -c -nc -q -t 0 --user-agent="" "http://losangeles.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/losangeles/
wget -c -nc -q -t 0 --user-agent="" "http://louisville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/louisville/
wget -c -nc -q -t 0 --user-agent="" "http://lubbock.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/lubbock/
wget -c -nc -q -t 0 --user-agent="" "http://lynchburg.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/lynchburg/
wget -c -nc -q -t 0 --user-agent="" "http://macon.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/macon/
wget -c -nc -q -t 0 --user-agent="" "http://madison.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/madison/
wget -c -nc -q -t 0 --user-agent="" "http://maine.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/maine/
wget -c -nc -q -t 0 --user-agent="" "http://mankato.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/mankato/
wget -c -nc -q -t 0 --user-agent="" "http://mansfield.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/mansfield/
wget -c -nc -q -t 0 --user-agent="" "http://martinsburg.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/martinsburg/
wget -c -nc -q -t 0 --user-agent="" "http://mcallen.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/mcallen/
wget -c -nc -q -t 0 --user-agent="" "http://medford.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/medford/
wget -c -nc -q -t 0 --user-agent="" "http://memphis.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/memphis/
wget -c -nc -q -t 0 --user-agent="" "http://mendocino.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/mendocino/
wget -c -nc -q -t 0 --user-agent="" "http://merced.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/merced/
wget -c -nc -q -t 0 --user-agent="" "http://miami.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/miami/
wget -c -nc -q -t 0 --user-agent="" "http://milwaukee.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/milwaukee/
wget -c -nc -q -t 0 --user-agent="" "http://minneapolis.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/minneapolis/
wget -c -nc -q -t 0 --user-agent="" "http://mobile.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/mobile/
wget -c -nc -q -t 0 --user-agent="" "http://modesto.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/modesto/
wget -c -nc -q -t 0 --user-agent="" "http://mohave.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/mohave/
wget -c -nc -q -t 0 --user-agent="" "http://monroe.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/monroe/
wget -c -nc -q -t 0 --user-agent="" "http://montana.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/montana/
wget -c -nc -q -t 0 --user-agent="" "http://monterey.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/monterey/
wget -c -nc -q -t 0 --user-agent="" "http://montgomery.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/montgomery/
wget -c -nc -q -t 0 --user-agent="" "http://morgantown.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/morgantown/
wget -c -nc -q -t 0 --user-agent="" "http://muncie.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/muncie/
wget -c -nc -q -t 0 --user-agent="" "http://muskegon.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/muskegon/
wget -c -nc -q -t 0 --user-agent="" "http://myrtlebeach.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/myrtlebeach/
wget -c -nc -q -t 0 --user-agent="" "http://nashville.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/nashville/
wget -c -nc -q -t 0 --user-agent="" "http://nd.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/nd/
wget -c -nc -q -t 0 --user-agent="" "http://newhaven.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/newhaven/
wget -c -nc -q -t 0 --user-agent="" "http://newjersey.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/newjersey/
wget -c -nc -q -t 0 --user-agent="" "http://newlondon.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/newlondon/
wget -c -nc -q -t 0 --user-agent="" "http://neworleans.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/neworleans/
wget -c -nc -q -t 0 --user-agent="" "http://newyork.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/newyork/
wget -c -nc -q -t 0 --user-agent="" "http://nh.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/nh/
wget -c -nc -q -t 0 --user-agent="" "http://nmi.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/nmi/
wget -c -nc -q -t 0 --user-agent="" "http://norfolk.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/norfolk/
wget -c -nc -q -t 0 --user-agent="" "http://northmiss.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/northmiss/
wget -c -nc -q -t 0 --user-agent="" "http://nwct.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/nwct/
wget -c -nc -q -t 0 --user-agent="" "http://ocala.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/ocala/
wget -c -nc -q -t 0 --user-agent="" "http://odessa.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/odessa/
wget -c -nc -q -t 0 --user-agent="" "http://ogden.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/ogden/
wget -c -nc -q -t 0 --user-agent="" "http://oklahomacity.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/oklahomacity/
wget -c -nc -q -t 0 --user-agent="" "http://olympic.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/olympic/
wget -c -nc -q -t 0 --user-agent="" "http://omaha.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/omaha/
wget -c -nc -q -t 0 --user-agent="" "http://orangecounty.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/orangecounty/
wget -c -nc -q -t 0 --user-agent="" "http://oregoncoast.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/oregoncoast/
wget -c -nc -q -t 0 --user-agent="" "http://orlando.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/orlando/
wget -c -nc -q -t 0 --user-agent="" "http://outerbanks.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/outerbanks/
wget -c -nc -q -t 0 --user-agent="" "http://palmsprings.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/palmsprings/
wget -c -nc -q -t 0 --user-agent="" "http://panamacity.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/panamacity/
wget -c -nc -q -t 0 --user-agent="" "http://parkersburg.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/parkersburg/
wget -c -nc -q -t 0 --user-agent="" "http://pennstate.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/pennstate/
wget -c -nc -q -t 0 --user-agent="" "http://pensacola.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/pensacola/
wget -c -nc -q -t 0 --user-agent="" "http://peoria.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/peoria/
wget -c -nc -q -t 0 --user-agent="" "http://philadelphia.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/philadelphia/
wget -c -nc -q -t 0 --user-agent="" "http://phoenix.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/phoenix/
wget -c -nc -q -t 0 --user-agent="" "http://pittsburgh.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/pittsburgh/
wget -c -nc -q -t 0 --user-agent="" "http://plattsburgh.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/plattsburgh/
wget -c -nc -q -t 0 --user-agent="" "http://poconos.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/poconos/
wget -c -nc -q -t 0 --user-agent="" "http://porthuron.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/porthuron/
wget -c -nc -q -t 0 --user-agent="" "http://portland.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/portland/
wget -c -nc -q -t 0 --user-agent="" "http://prescott.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/prescott/
wget -c -nc -q -t 0 --user-agent="" "http://providence.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/providence/
wget -c -nc -q -t 0 --user-agent="" "http://provo.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/provo/
wget -c -nc -q -t 0 --user-agent="" "http://pueblo.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/pueblo/
wget -c -nc -q -t 0 --user-agent="" "http://pullman.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/pullman/
wget -c -nc -q -t 0 --user-agent="" "http://quadcities.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/quadcities/
wget -c -nc -q -t 0 --user-agent="" "http://racine.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/racine/
wget -c -nc -q -t 0 --user-agent="" "http://raleigh.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/raleigh/
wget -c -nc -q -t 0 --user-agent="" "http://reading.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/reading/
wget -c -nc -q -t 0 --user-agent="" "http://redding.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/redding/
wget -c -nc -q -t 0 --user-agent="" "http://reno.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/reno/
wget -c -nc -q -t 0 --user-agent="" "http://richmond.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/richmond/
wget -c -nc -q -t 0 --user-agent="" "http://rmn.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/rmn/
wget -c -nc -q -t 0 --user-agent="" "http://roanoke.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/roanoke/
wget -c -nc -q -t 0 --user-agent="" "http://rochester.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/rochester/
wget -c -nc -q -t 0 --user-agent="" "http://rockford.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/rockford/
wget -c -nc -q -t 0 --user-agent="" "http://rockies.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/rockies/
wget -c -nc -q -t 0 --user-agent="" "http://roseburg.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/roseburg/
wget -c -nc -q -t 0 --user-agent="" "http://roswell.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/roswell/
wget -c -nc -q -t 0 --user-agent="" "http://sacramento.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/sacramento/
wget -c -nc -q -t 0 --user-agent="" "http://saginaw.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/saginaw/
wget -c -nc -q -t 0 --user-agent="" "http://salem.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/salem/
wget -c -nc -q -t 0 --user-agent="" "http://saltlakecity.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/saltlakecity/
wget -c -nc -q -t 0 --user-agent="" "http://sanantonio.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/sanantonio/
wget -c -nc -q -t 0 --user-agent="" "http://sandiego.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/sandiego/
wget -c -nc -q -t 0 --user-agent="" "http://sandusky.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/sandusky/
wget -c -nc -q -t 0 --user-agent="" "http://sanmarcos.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/sanmarcos/
wget -c -nc -q -t 0 --user-agent="" "http://santabarbara.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/santabarbara/
wget -c -nc -q -t 0 --user-agent="" "http://santafe.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/santafe/
wget -c -nc -q -t 0 --user-agent="" "http://sarasota.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/sarasota/
wget -c -nc -q -t 0 --user-agent="" "http://savannah.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/savannah/
wget -c -nc -q -t 0 --user-agent="" "http://scranton.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/scranton/
wget -c -nc -q -t 0 --user-agent="" "http://sd.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/sd/
wget -c -nc -q -t 0 --user-agent="" "http://seattle.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/seattle/
wget -c -nc -q -t 0 --user-agent="" "http://semo.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/semo/
wget -c -nc -q -t 0 --user-agent="" "http://sfbay.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/sfbay/
wget -c -nc -q -t 0 --user-agent="" "http://sheboygan.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/sheboygan/
wget -c -nc -q -t 0 --user-agent="" "http://shoals.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/shoals/
wget -c -nc -q -t 0 --user-agent="" "http://shreveport.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/shreveport/
wget -c -nc -q -t 0 --user-agent="" "http://sierravista.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/sierravista/
wget -c -nc -q -t 0 --user-agent="" "http://siouxcity.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/siouxcity/
wget -c -nc -q -t 0 --user-agent="" "http://skagit.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/skagit/
wget -c -nc -q -t 0 --user-agent="" "http://slo.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/slo/
wget -c -nc -q -t 0 --user-agent="" "http://smd.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/smd/
wget -c -nc -q -t 0 --user-agent="" "http://southbend.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/southbend/
wget -c -nc -q -t 0 --user-agent="" "http://southcoast.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/southcoast/
wget -c -nc -q -t 0 --user-agent="" "http://southjersey.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/southjersey/
wget -c -nc -q -t 0 --user-agent="" "http://spacecoast.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/spacecoast/
wget -c -nc -q -t 0 --user-agent="" "http://spokane.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/spokane/
wget -c -nc -q -t 0 --user-agent="" "http://springfield.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/springfield/
wget -c -nc -q -t 0 --user-agent="" "http://springfieldil.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/springfieldil/
wget -c -nc -q -t 0 --user-agent="" "http://staugustine.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/staugustine/
wget -c -nc -q -t 0 --user-agent="" "http://stcloud.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/stcloud/
wget -c -nc -q -t 0 --user-agent="" "http://stgeorge.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/stgeorge/
wget -c -nc -q -t 0 --user-agent="" "http://stillwater.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/stillwater/
wget -c -nc -q -t 0 --user-agent="" "http://stlouis.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/stlouis/
wget -c -nc -q -t 0 --user-agent="" "http://stockton.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/stockton/
wget -c -nc -q -t 0 --user-agent="" "http://swmi.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/swmi/
wget -c -nc -q -t 0 --user-agent="" "http://syracuse.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/syracuse/
wget -c -nc -q -t 0 --user-agent="" "http://tallahassee.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/tallahassee/
wget -c -nc -q -t 0 --user-agent="" "http://tampa.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/tampa/
wget -c -nc -q -t 0 --user-agent="" "http://terrahaute.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/terrahaute/
wget -c -nc -q -t 0 --user-agent="" "http://texarkana.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/texarkana/
wget -c -nc -q -t 0 --user-agent="" "http://tippecanoe.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/tippecanoe/
wget -c -nc -q -t 0 --user-agent="" "http://toledo.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/toledo/
wget -c -nc -q -t 0 --user-agent="" "http://topeka.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/topeka/
wget -c -nc -q -t 0 --user-agent="" "http://treasure.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/treasure/
wget -c -nc -q -t 0 --user-agent="" "http://tricities.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/tricities/
wget -c -nc -q -t 0 --user-agent="" "http://tucson.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/tucson/
wget -c -nc -q -t 0 --user-agent="" "http://tulsa.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/tulsa/
wget -c -nc -q -t 0 --user-agent="" "http://tuscaloosa.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/tuscaloosa/
wget -c -nc -q -t 0 --user-agent="" "http://twinfalls.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/twinfalls/
wget -c -nc -q -t 0 --user-agent="" "http://up.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/up/
wget -c -nc -q -t 0 --user-agent="" "http://utica.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/utica/
wget -c -nc -q -t 0 --user-agent="" "http://valdosta.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/valdosta/
wget -c -nc -q -t 0 --user-agent="" "http://ventura.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/ventura/
wget -c -nc -q -t 0 --user-agent="" "http://victoriatx.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/victoriatx/
wget -c -nc -q -t 0 --user-agent="" "http://visalia.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/visalia/
wget -c -nc -q -t 0 --user-agent="" "http://waco.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/waco/
wget -c -nc -q -t 0 --user-agent="" "http://washingtondc.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/washingtondc/
wget -c -nc -q -t 0 --user-agent="" "http://waterloo.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/waterloo/
wget -c -nc -q -t 0 --user-agent="" "http://watertown.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/watertown/
wget -c -nc -q -t 0 --user-agent="" "http://wausau.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/wausau/
wget -c -nc -q -t 0 --user-agent="" "http://wenatchee.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/wenatchee/
wget -c -nc -q -t 0 --user-agent="" "http://westernmass.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/westernmass/
wget -c -nc -q -t 0 --user-agent="" "http://westky.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/westky/
wget -c -nc -q -t 0 --user-agent="" "http://westmd.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/westmd/
wget -c -nc -q -t 0 --user-agent="" "http://westslope.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/westslope/
wget -c -nc -q -t 0 --user-agent="" "http://wheeling.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/wheeling/
wget -c -nc -q -t 0 --user-agent="" "http://wichita.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/wichita/
wget -c -nc -q -t 0 --user-agent="" "http://wichitafalls.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/wichitafalls/
wget -c -nc -q -t 0 --user-agent="" "http://williamsport.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/williamsport/
wget -c -nc -q -t 0 --user-agent="" "http://wilmington.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/wilmington/
wget -c -nc -q -t 0 --user-agent="" "http://winstonsalem.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/winstonsalem/
wget -c -nc -q -t 0 --user-agent="" "http://worcester.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/worcester/
wget -c -nc -q -t 0 --user-agent="" "http://wv.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/wv/
wget -c -nc -q -t 0 --user-agent="" "http://wyoming.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/wyoming/
wget -c -nc -q -t 0 --user-agent="" "http://yakima.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/yakima/
wget -c -nc -q -t 0 --user-agent="" "http://york.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/york/
wget -c -nc -q -t 0 --user-agent="" "http://youngstown.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/youngstown/
wget -c -nc -q -t 0 --user-agent="" "http://yubasutter.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/yubasutter/
wget -c -nc -q -t 0 --user-agent="" "http://yuma.craigslist.org/search/ers/?query=w4m&format=rss" -P cities/yuma/
echo "RSS download complete."
echo "Obtaining post URLs."
post()
{     while read line1; do 
          mv cities/$line1/index* cities/$line1/rsspost
          cat cities/$line1/rsspost | grep resource > cities/$line1/posts.url
          mkdir data/$line1
          mkdir phlist/$line1
     done
}
ls -1 cities/ | post
replace()
{     while read line1; do 
          rpl -q "  <rdf:li rdf:resource=" "" cities/$line1/posts.url
          rpl -q " />" "" cities/$line1/posts.url
          rpl -e -q "\042" "" cities/$line1/posts.url
     done
}
ls -1 cities/ | replace
echo "Downloading posts."
get()
{     while read line1; do 
          wget -c -q -t 0 --user-agent="" -P cities/$line1/ -i cities/$line1/posts.url
     done
}
ls -1 cities/ | get
echo "Download complete."
datadump()
{     while read line1; do
          cat cities/$line1/*.html > data/$line1/rawdata
     done
}
ls -1 cities/ | datadump
dump()
{     while read line1; do
          links -dump -no-references -no-numbering data/$line1/rawdata > data/$line1/rawdata.txt
     done
}
ls -1 cities/ | dump
echo "Tailoring data."
rpl -q -i "zero" "0" data/*/rawdata.txt
rpl -q -i "one" "1" data/*/rawdata.txt
rpl -q -i "two" "2" data/*/rawdata.txt
rpl -q -i "three" "3" data/*/rawdata.txt
rpl -q -i "four" "4" data/*/rawdata.txt
rpl -q -i "five" "5" data/*/rawdata.txt
rpl -q -i "six" "6" data/*/rawdata.txt
rpl -q -i "seven" "7" data/*/rawdata.txt
rpl -q -i "eight" "8" data/*/rawdata.txt
rpl -q -i "nine" "9" data/*/rawdata.txt
rpl -R -q -e '\041' " " data/*/rawdata.txt
rpl -R -q -e '\042' " " data/*/rawdata.txt
rpl -R -q -e '\043' " " data/*/rawdata.txt
rpl -R -q -e '\044' " " data/*/rawdata.txt
rpl -R -q -e '\045' " " data/*/rawdata.txt
rpl -R -q -e '\046' " " data/*/rawdata.txt
rpl -R -q -e '\047' " " data/*/rawdata.txt
rpl -R -q -e '\048' " " data/*/rawdata.txt
rpl -R -q -e '\049' " " data/*/rawdata.txt
rpl -R -q -e '\050' " " data/*/rawdata.txt
rpl -R -q -e '\051' " " data/*/rawdata.txt
rpl -R -q -e '\052' " " data/*/rawdata.txt
rpl -R -q -e '\053' " " data/*/rawdata.txt
rpl -R -q -e '\054' " " data/*/rawdata.txt
rpl -R -q -e '\055' " " data/*/rawdata.txt
rpl -R -q -e '\056' " " data/*/rawdata.txt
rpl -R -q -e '\057' " " data/*/rawdata.txt
rpl -R -q -e '\058' " " data/*/rawdata.txt
rpl -R -q -e '\059' " " data/*/rawdata.txt
rpl -R -q -e '\072' " " data/*/rawdata.txt
rpl -R -q -e '\073' " " data/*/rawdata.txt
rpl -R -q -e '\074' " " data/*/rawdata.txt
rpl -R -q -e '\075' " " data/*/rawdata.txt
rpl -R -q -e '\076' " " data/*/rawdata.txt
rpl -R -q -e '\077' " " data/*/rawdata.txt
rpl -R -q -e '\078' " " data/*/rawdata.txt
rpl -R -q -e '\079' " " data/*/rawdata.txt
rpl -R -q -e '\080' " " data/*/rawdata.txt
rpl -R -q -e '\081' " " data/*/rawdata.txt
rpl -R -q -e '\082' " " data/*/rawdata.txt
rpl -R -q -e '\083' " " data/*/rawdata.txt
rpl -R -q -e '\084' " " data/*/rawdata.txt
rpl -R -q -e '\085' " " data/*/rawdata.txt
rpl -R -q -e '\086' " " data/*/rawdata.txt
rpl -R -q -e '\087' " " data/*/rawdata.txt
rpl -R -q -e '\088' " " data/*/rawdata.txt
rpl -R -q -e '\089' " " data/*/rawdata.txt
rpl -R -q -e '\090' " " data/*/rawdata.txt
rpl -R -q -e '\091' " " data/*/rawdata.txt
rpl -R -q -e '\092' " " data/*/rawdata.txt
rpl -R -q -e '\093' " " data/*/rawdata.txt
rpl -R -q -e '\094' " " data/*/rawdata.txt
rpl -R -q -e '\095' " " data/*/rawdata.txt
rpl -R -q -e '\096' " " data/*/rawdata.txt
rpl -R -q -e '\097' " " data/*/rawdata.txt
rpl -R -q -e '\098' " " data/*/rawdata.txt
rpl -R -q -e '\099' " " data/*/rawdata.txt
rpl -R -q -e '\100' " " data/*/rawdata.txt
rpl -R -q -e '\101' " " data/*/rawdata.txt
rpl -R -q -e '\102' " " data/*/rawdata.txt
rpl -R -q -e '\103' " " data/*/rawdata.txt
rpl -R -q -e '\104' " " data/*/rawdata.txt
rpl -R -q -e '\105' " " data/*/rawdata.txt
rpl -R -q -e '\106' " " data/*/rawdata.txt
rpl -R -q -e '\107' " " data/*/rawdata.txt
rpl -R -q -e '\108' " " data/*/rawdata.txt
rpl -R -q -e '\109' " " data/*/rawdata.txt
rpl -R -q -e '\110' " " data/*/rawdata.txt
rpl -R -q -e '\111' " " data/*/rawdata.txt
rpl -R -q -e '\112' " " data/*/rawdata.txt
rpl -R -q -e '\113' " " data/*/rawdata.txt
rpl -R -q -e '\114' " " data/*/rawdata.txt
rpl -R -q -e '\115' " " data/*/rawdata.txt
rpl -R -q -e '\116' " " data/*/rawdata.txt
rpl -R -q -e '\117' " " data/*/rawdata.txt
rpl -R -q -e '\118' " " data/*/rawdata.txt
rpl -R -q -e '\119' " " data/*/rawdata.txt
rpl -R -q -e '\120' " " data/*/rawdata.txt
rpl -R -q -e '\121' " " data/*/rawdata.txt
rpl -R -q -e '\122' " " data/*/rawdata.txt
rpl -R -q -e '\123' " " data/*/rawdata.txt
rpl -R -q -e '\124' " " data/*/rawdata.txt
rpl -R -q -e '\125' " " data/*/rawdata.txt
rpl -R -q -e '\126' " " data/*/rawdata.txt
rpl -R -q -e '\127' " " data/*/rawdata.txt
rpl -R -q -e '\128' " " data/*/rawdata.txt
rpl -R -q -e '\129' " " data/*/rawdata.txt
rpl -R -q -e '\130' " " data/*/rawdata.txt
rpl -R -q -e '\131' " " data/*/rawdata.txt
rpl -R -q -e '\132' " " data/*/rawdata.txt
rpl -R -q -e '\133' " " data/*/rawdata.txt
rpl -R -q -e '\134' " " data/*/rawdata.txt
rpl -R -q -e '\135' " " data/*/rawdata.txt
rpl -R -q -e '\136' " " data/*/rawdata.txt
rpl -R -q -e '\137' " " data/*/rawdata.txt
rpl -R -q -e '\138' " " data/*/rawdata.txt
rpl -R -q -e '\139' " " data/*/rawdata.txt
rpl -R -q -e '\140' " " data/*/rawdata.txt
rpl -R -q -e '\141' " " data/*/rawdata.txt
rpl -R -q -e '\142' " " data/*/rawdata.txt
rpl -R -q -e '\143' " " data/*/rawdata.txt
rpl -R -q -e '\144' " " data/*/rawdata.txt
rpl -R -q -e '\145' " " data/*/rawdata.txt
rpl -R -q -e '\146' " " data/*/rawdata.txt
rpl -R -q -e '\147' " " data/*/rawdata.txt
rpl -R -q -e '\148' " " data/*/rawdata.txt
rpl -R -q -e '\149' " " data/*/rawdata.txt
rpl -R -q -e '\150' " " data/*/rawdata.txt
rpl -R -q -e '\151' " " data/*/rawdata.txt
rpl -R -q -e '\152' " " data/*/rawdata.txt
rpl -R -q -e '\153' " " data/*/rawdata.txt
rpl -R -q -e '\154' " " data/*/rawdata.txt
rpl -R -q -e '\155' " " data/*/rawdata.txt
rpl -R -q -e '\156' " " data/*/rawdata.txt
rpl -R -q -e '\157' " " data/*/rawdata.txt
rpl -R -q -e '\158' " " data/*/rawdata.txt
rpl -R -q -e '\159' " " data/*/rawdata.txt
rpl -R -q -e '\160' " " data/*/rawdata.txt
rpl -R -q -e '\161' " " data/*/rawdata.txt
rpl -R -q -e '\162' " " data/*/rawdata.txt
rpl -R -q -e '\163' " " data/*/rawdata.txt
rpl -R -q -e '\164' " " data/*/rawdata.txt
rpl -R -q -e '\165' " " data/*/rawdata.txt
rpl -R -q -e '\166' " " data/*/rawdata.txt
rpl -R -q -e '\167' " " data/*/rawdata.txt
rpl -R -q -e '\168' " " data/*/rawdata.txt
rpl -R -q -e '\169' " " data/*/rawdata.txt
rpl -R -q -e '\170' " " data/*/rawdata.txt
rpl -R -q -e '\171' " " data/*/rawdata.txt
rpl -R -q -e '\172' " " data/*/rawdata.txt
rpl -R -q -e '\173' " " data/*/rawdata.txt
rpl -R -q -e '\174' " " data/*/rawdata.txt
rpl -R -q -e '\175' " " data/*/rawdata.txt
rpl -R -q -e '\176' " " data/*/rawdata.txt
rpl -R -q -e '\200' " " data/*/rawdata.txt
rpl -R -q -e '\201' " " data/*/rawdata.txt
rpl -R -q -e '\202' " " data/*/rawdata.txt
rpl -R -q -e '\203' " " data/*/rawdata.txt
rpl -R -q -e '\204' " " data/*/rawdata.txt
rpl -R -q -e '\205' " " data/*/rawdata.txt
rpl -R -q -e '\206' " " data/*/rawdata.txt
rpl -R -q -e '\207' " " data/*/rawdata.txt
rpl -R -q -e '\208' " " data/*/rawdata.txt
rpl -R -q -e '\209' " " data/*/rawdata.txt
rpl -R -q -e '\210' " " data/*/rawdata.txt
rpl -R -q -e '\211' " " data/*/rawdata.txt
rpl -R -q -e '\212' " " data/*/rawdata.txt
rpl -R -q -e '\213' " " data/*/rawdata.txt
rpl -R -q -e '\214' " " data/*/rawdata.txt
rpl -R -q -e '\215' " " data/*/rawdata.txt
rpl -R -q -e '\216' " " data/*/rawdata.txt
rpl -R -q -e '\217' " " data/*/rawdata.txt
rpl -R -q -e '\218' " " data/*/rawdata.txt
rpl -R -q -e '\219' " " data/*/rawdata.txt
rpl -R -q -e '\220' " " data/*/rawdata.txt
rpl -R -q -e '\221' " " data/*/rawdata.txt
rpl -R -q -e '\222' " " data/*/rawdata.txt
rpl -R -q -e '\223' " " data/*/rawdata.txt
rpl -R -q -e '\224' " " data/*/rawdata.txt
rpl -R -q -e '\225' " " data/*/rawdata.txt
rpl -R -q -e '\226' " " data/*/rawdata.txt
rpl -R -q -e '\227' " " data/*/rawdata.txt
rpl -R -q -e '\228' " " data/*/rawdata.txt
rpl -R -q -e '\229' " " data/*/rawdata.txt
rpl -R -q -e '\230' " " data/*/rawdata.txt
rpl -R -q -e '\231' " " data/*/rawdata.txt
rpl -R -q -e '\232' " " data/*/rawdata.txt
rpl -R -q -e '\233' " " data/*/rawdata.txt
rpl -R -q -e '\234' " " data/*/rawdata.txt
rpl -R -q -e '\235' " " data/*/rawdata.txt
rpl -R -q -e '\236' " " data/*/rawdata.txt
rpl -R -q -e '\237' " " data/*/rawdata.txt
rpl -R -q -e '\238' " " data/*/rawdata.txt
rpl -R -q -e '\239' " " data/*/rawdata.txt
rpl -R -q -e '\240' " " data/*/rawdata.txt
rpl -R -q -e '\241' " " data/*/rawdata.txt
rpl -R -q -e '\242' " " data/*/rawdata.txt
rpl -R -q -e '\243' " " data/*/rawdata.txt
rpl -R -q -e '\244' " " data/*/rawdata.txt
rpl -R -q -e '\245' " " data/*/rawdata.txt
rpl -R -q -e '\246' " " data/*/rawdata.txt
rpl -R -q -e '\247' " " data/*/rawdata.txt
rpl -R -q -e '\248' " " data/*/rawdata.txt
rpl -R -q -e '\249' " " data/*/rawdata.txt
rpl -R -q -e '\250' " " data/*/rawdata.txt
rpl -R -q -e '\251' " " data/*/rawdata.txt
rpl -R -q -e '\252' " " data/*/rawdata.txt
rpl -R -q -e '\253' " " data/*/rawdata.txt
rpl -R -q -e '\254' " " data/*/rawdata.txt
rpl -R -q -e '\255' " " data/*/rawdata.txt
rpl -R -q -e '\256' " " data/*/rawdata.txt
rpl -R -q -e '\257' " " data/*/rawdata.txt
rpl -R -q -e '\258' " " data/*/rawdata.txt
rpl -R -q -e '\259' " " data/*/rawdata.txt
rpl -R -q -e '\260' " " data/*/rawdata.txt
rpl -R -q -e '\261' " " data/*/rawdata.txt
rpl -R -q -e '\262' " " data/*/rawdata.txt
rpl -R -q -e '\263' " " data/*/rawdata.txt
rpl -R -q -e '\264' " " data/*/rawdata.txt
rpl -R -q -e '\265' " " data/*/rawdata.txt
rpl -R -q -e '\266' " " data/*/rawdata.txt
rpl -R -q -e '\267' " " data/*/rawdata.txt
rpl -R -q -e '\268' " " data/*/rawdata.txt
rpl -R -q -e '\269' " " data/*/rawdata.txt
rpl -R -q -e '\270' " " data/*/rawdata.txt
rpl -R -q -e '\271' " " data/*/rawdata.txt
rpl -R -q -e '\272' " " data/*/rawdata.txt
rpl -R -q -e '\273' " " data/*/rawdata.txt
rpl -R -q -e '\274' " " data/*/rawdata.txt
rpl -R -q -e '\275' " " data/*/rawdata.txt
rpl -R -q -e '\276' " " data/*/rawdata.txt
rpl -R -q -e '\277' " " data/*/rawdata.txt
rpl -R -q -e '\278' " " data/*/rawdata.txt
rpl -R -q -e '\279' " " data/*/rawdata.txt
rpl -R -q -e '\280' " " data/*/rawdata.txt
rpl -R -q -e '\281' " " data/*/rawdata.txt
rpl -R -q -e '\282' " " data/*/rawdata.txt
rpl -R -q -e '\283' " " data/*/rawdata.txt
rpl -R -q -e '\284' " " data/*/rawdata.txt
rpl -R -q -e '\285' " " data/*/rawdata.txt
rpl -R -q -e '\286' " " data/*/rawdata.txt
rpl -R -q -e '\287' " " data/*/rawdata.txt
rpl -R -q -e '\288' " " data/*/rawdata.txt
rpl -R -q -e '\289' " " data/*/rawdata.txt
rpl -R -q -e '\290' " " data/*/rawdata.txt
rpl -R -q -e '\291' " " data/*/rawdata.txt
rpl -R -q -e '\292' " " data/*/rawdata.txt
rpl -R -q -e '\293' " " data/*/rawdata.txt
rpl -R -q -e '\294' " " data/*/rawdata.txt
rpl -R -q -e '\295' " " data/*/rawdata.txt
rpl -R -q -e '\296' " " data/*/rawdata.txt
rpl -R -q -e '\297' " " data/*/rawdata.txt
rpl -R -q -e '\298' " " data/*/rawdata.txt
rpl -R -q -e '\299' " " data/*/rawdata.txt
rpl -R -q -e '\300' " " data/*/rawdata.txt
rpl -R -q -e '\301' " " data/*/rawdata.txt
rpl -R -q -e '\302' " " data/*/rawdata.txt
rpl -R -q -e '\303' " " data/*/rawdata.txt
rpl -R -q -e '\304' " " data/*/rawdata.txt
rpl -R -q -e '\305' " " data/*/rawdata.txt
rpl -R -q -e '\306' " " data/*/rawdata.txt
rpl -R -q -e '\307' " " data/*/rawdata.txt
rpl -R -q -e '\308' " " data/*/rawdata.txt
rpl -R -q -e '\309' " " data/*/rawdata.txt
rpl -R -q -e '\310' " " data/*/rawdata.txt
rpl -R -q -e '\311' " " data/*/rawdata.txt
rpl -R -q -e '\312' " " data/*/rawdata.txt
rpl -R -q -e '\313' " " data/*/rawdata.txt
rpl -R -q -e '\314' " " data/*/rawdata.txt
rpl -R -q -e '\315' " " data/*/rawdata.txt
rpl -R -q -e '\316' " " data/*/rawdata.txt
rpl -R -q -e '\317' " " data/*/rawdata.txt
rpl -R -q -e '\318' " " data/*/rawdata.txt
rpl -R -q -e '\319' " " data/*/rawdata.txt
rpl -R -q -e '\320' " " data/*/rawdata.txt
rpl -R -q -e '\321' " " data/*/rawdata.txt
rpl -R -q -e '\322' " " data/*/rawdata.txt
rpl -R -q -e '\323' " " data/*/rawdata.txt
rpl -R -q -e '\324' " " data/*/rawdata.txt
rpl -R -q -e '\325' " " data/*/rawdata.txt
rpl -R -q -e '\326' " " data/*/rawdata.txt
rpl -R -q -e '\327' " " data/*/rawdata.txt
rpl -R -q -e '\328' " " data/*/rawdata.txt
rpl -R -q -e '\329' " " data/*/rawdata.txt
rpl -R -q -e '\330' " " data/*/rawdata.txt
rpl -R -q -e '\331' " " data/*/rawdata.txt
rpl -R -q -e '\332' " " data/*/rawdata.txt
rpl -R -q -e '\333' " " data/*/rawdata.txt
rpl -R -q -e '\334' " " data/*/rawdata.txt
rpl -R -q -e '\335' " " data/*/rawdata.txt
rpl -R -q -e '\336' " " data/*/rawdata.txt
rpl -R -q -e '\337' " " data/*/rawdata.txt
rpl -R -q -e '\338' " " data/*/rawdata.txt
rpl -R -q -e '\339' " " data/*/rawdata.txt
rpl -R -q -e '\340' " " data/*/rawdata.txt
rpl -R -q -e '\341' " " data/*/rawdata.txt
rpl -R -q -e '\342' " " data/*/rawdata.txt
rpl -R -q -e '\343' " " data/*/rawdata.txt
rpl -R -q -e '\344' " " data/*/rawdata.txt
rpl -R -q -e '\345' " " data/*/rawdata.txt
rpl -R -q -e '\346' " " data/*/rawdata.txt
rpl -R -q -e '\347' " " data/*/rawdata.txt
rpl -R -q -e '\348' " " data/*/rawdata.txt
rpl -R -q -e '\349' " " data/*/rawdata.txt
rpl -R -q -e '\350' " " data/*/rawdata.txt
rpl -R -q -e '\351' " " data/*/rawdata.txt
rpl -R -q -e '\352' " " data/*/rawdata.txt
rpl -R -q -e '\353' " " data/*/rawdata.txt
rpl -R -q -e '\354' " " data/*/rawdata.txt
rpl -R -q -e '\355' " " data/*/rawdata.txt
rpl -R -q -e '\356' " " data/*/rawdata.txt
rpl -R -q -e '\357' " " data/*/rawdata.txt
rpl -R -q -e '\358' " " data/*/rawdata.txt
rpl -R -q -e '\359' " " data/*/rawdata.txt
rpl -R -q -e '\360' " " data/*/rawdata.txt
rpl -R -q -e '\361' " " data/*/rawdata.txt
rpl -R -q -e '\362' " " data/*/rawdata.txt
rpl -R -q -e '\363' " " data/*/rawdata.txt
rpl -R -q -e '\364' " " data/*/rawdata.txt
rpl -R -q -e '\365' " " data/*/rawdata.txt
rpl -R -q -e '\366' " " data/*/rawdata.txt
rpl -R -q -e '\367' " " data/*/rawdata.txt
rpl -R -q -e '\368' " " data/*/rawdata.txt
rpl -R -q -e '\369' " " data/*/rawdata.txt
rpl -R -q -e '\370' " " data/*/rawdata.txt
rpl -R -q -e '\371' " " data/*/rawdata.txt
rpl -R -q -e '\372' " " data/*/rawdata.txt
rpl -R -q -e '\373' " " data/*/rawdata.txt
rpl -R -q -e '\374' " " data/*/rawdata.txt
rpl -R -q -e '\375' " " data/*/rawdata.txt
rpl -R -q -e '\376' " " data/*/rawdata.txt
rpl -R -q -e '\377' " " data/*/rawdata.txt
rpl -R -q " " "" data/*/rawdata.txt
tailor2()
{     while read line1; do
          sort -u -n data/$line1/rawdata.txt | grep "^[[:digit:]]\{10\}$" | awk '{if ($1 > "2011000000") print $1}' > phlist/$line1/"`date +%Y%m%d_%H%M`"
     done
}
ls -1 cities/ | tailor2
echo "Raw data output complete. Starting KML file generation."
find phlist -mindepth 2 -exec grep -H [[:digit:]] {} \; | column -t -s"/"":" | awk '{print $4"\t"$3"\t"$2"_"}' | sort -n | awk '
{sub ("\tabilene_", "\tabilene\t-99.683333333333,32.416666666667");
sub ("\takroncanton_", "\takroncanton\t-81.433333333333,40.916666666667");
sub ("\talbany_", "\talbany\t-73.800000000000,42.750000000000");
sub ("\talbuquerque_", "\talbuquerque\t-106.616666666667,35.050000000000");
sub ("\tallentown_", "\tallentown\t-75.433333333333,40.650000000000");
sub ("\taltoona_", "\taltoona\t-78.316666666667,40.300000000000");
sub ("\tamarillo_", "\tamarillo\t-100.700000000000,35.233333333333");
sub ("\tames_", "\tames\t-93.800000000000,42.033333333333");
sub ("\tanchorage_", "\tanchorage\t-150.016666666667,61.166666666667");
sub ("\tannapolis_", "\tannapolis\t-76.501158000000,38.972944000000");
sub ("\tannarbor_", "\tannarbor\t-83.748333333333,42.281388888889");
sub ("\tappleton_", "\tappleton\t-88.383333333333,44.250000000000");
sub ("\tasheville_", "\tasheville\t-82.533333333333,35.433333333333");
sub ("\tathensga_", "\tathensga\t-83.316666666667,33.950000000000");
sub ("\tathensohio_", "\tathensohio\t-82.100000000000,39.333333333333");
sub ("\tatlanta_", "\tatlanta\t-84.433333333333,33.650000000000");
sub ("\tauburn_", "\tauburn\t-85.500000000000,32.600000000000");
sub ("\taugusta_", "\taugusta\t-81.966666666667,33.366666666667");
sub ("\taustin_", "\taustin\t-97.700000000000,30.300000000000");
sub ("\tbakersfield_", "\tbakersfield\t-119.050000000000,35.416666666667");
sub ("\tbaltimore_", "\tbaltimore\t-76.666666666667,39.183333333333");
sub ("\tbatonrouge_", "\tbatonrouge\t-91.150000000000,30.533333333333");
sub ("\tbeaumont_", "\tbeaumont\t-94.016666666667,29.950000000000");
sub ("\tbellingham_", "\tbellingham\t-122.533333333333,48.800000000000");
sub ("\tbend_", "\tbend\t-121.316666666667,44.066666666667");
sub ("\tbgky_", "\tbgky\t-86.466666666667,35.966666666667");
sub ("\tbham_", "\tbham\t-86.750000000000,33.566666666667");
sub ("\tbinghamton_", "\tbinghamton\t-75.983333333333,42.216666666667");
sub ("\tblacksburg_", "\tblacksburg\t-80.417778000000,37.230000000000");
sub ("\tbloomington_", "\tbloomington\t-86.616666666667,39.133333333333");
sub ("\tbn_", "\tbn\t-88.950000000000,40.483333333333");
sub ("\tboise_", "\tboise\t-116.216666666667,43.566666666667");
sub ("\tboone_", "\tboone\t-81.668611000000,36.211389000000");
sub ("\tboston_", "\tboston\t-71.033333333333,42.366666666667");
sub ("\tboulder_", "\tboulder\t-105.266666666667,40.000000000000");
sub ("\tbrownsville_", "\tbrownsville\t-97.433333333333,25.900000000000");
sub ("\tbrunswick_", "\tbrunswick\t-81.483333333333,31.250000000000");
sub ("\tbuffalo_", "\tbuffalo\t-78.733333333333,42.933333333333");
sub ("\tburlington_", "\tburlington\t-73.150000000000,44.466666666667");
sub ("\tcapecod_", "\tcapecod\t-70.296940000000,41.688890000000");
sub ("\tcarbondale_", "\tcarbondale\t-89.250000000000,37.783333333333");
sub ("\tcatskills_", "\tcatskills\t-74.386389000000,41.998611000000");
sub ("\tcedarrapids_", "\tcedarrapids\t-91.700000000000,41.883333333333");
sub ("\tcentralmich_", "\tcentralmich\t-84.766666666667,43.583333333333");
sub ("\tchambana_", "\tchambana\t-88.283333333333,40.033333333333");
sub ("\tcharleston_", "\tcharleston\t-80.033333333333,32.900000000000");
sub ("\tcharlestonwv_", "\tcharlestonwv\t-81.600000000000,38.366666666667");
sub ("\tcharlotte_", "\tcharlotte\t-80.933333333333,35.216666666667");
sub ("\tcharlottesville_", "\tcharlottesville\t-78.516666666667,38.033333333333");
sub ("\tchattanooga_", "\tchattanooga\t-85.200000000000,35.033333333333");
sub ("\tchautauqua_", "\tchautauqua\t-79.410000000000,42.300000000000");
sub ("\tchicago_", "\tchicago\t-87.633333333333,41.883333333333");
sub ("\tchico_", "\tchico\t-121.850000000000,39.800000000000");
sub ("\tcincinnati_", "\tcincinnati\t-84.516666666667,39.150000000000");
sub ("\tclarksville_", "\tclarksville\t-87.366666666667,36.550000000000");
sub ("\tcleveland_", "\tcleveland\t-81.850000000000,41.400000000000");
sub ("\tcnj_", "\tcnj\t-74.756111000000,40.221667000000");
sub ("\tcollegestation_", "\tcollegestation\t-96.314444000000,30.601389000000");
sub ("\tcolumbia_", "\tcolumbia\t-81.116666666667,33.950000000000");
sub ("\tcolumbiamo_", "\tcolumbiamo\t-92.366666666667,38.966666666667");
sub ("\tcolumbus_", "\tcolumbus\t-82.883333333333,40.000000000000");
sub ("\tcolumbusga_", "\tcolumbusga\t-84.933333333333,32.516666666667");
sub ("\tcorpuschristi_", "\tcorpuschristi\t-97.500000000000,27.766666666667");
sub ("\tcorvallis_", "\tcorvallis\t-123.283333333333,44.500000000000");
sub ("\tcosprings_", "\tcosprings\t-104.716666666667,38.816666666667");
sub ("\tdallas_", "\tdallas\t-96.850000000000,32.850000000000");
sub ("\tdanville_", "\tdanville\t-79.333333333333,36.566666666667");
sub ("\tdayton_", "\tdaytona\t-81.050000000000,29.183333333333");
sub ("\tdaytona_", "\tdayton\t-84.216666666667,39.900000000000");
sub ("\tdecatur_", "\tdecatur\t-88.866666666667,39.833333333333");
sub ("\tdelaware_", "\tdelaware\t-75.466666666667,39.133333333333");
sub ("\tdenver_", "\tdenver\t-104.866666666667,39.750000000000");
sub ("\tdesmoines_", "\tdesmoines\t-93.650000000000,41.533333333333");
sub ("\tdetroit_", "\tdetroit\t-83.016666666667,42.416666666667");
sub ("\tdothan_", "\tdothan\t-85.450000000000,31.316666666667");
sub ("\tdubuque_", "\tdubuque\t-90.700000000000,42.400000000000");
sub ("\tduluth_", "\tduluth\t-92.183333333333,46.833333333333");
sub ("\teasternshore_", "\teasternshore\t-75.593333000000,38.365833000000");
sub ("\teastidaho_", "\teastidaho\t-112.032500000000,43.491667000000");
sub ("\teastnc_", "\teastnc\t-77.372500000000,35.601667000000");
sub ("\teastoregon_", "\teastoregon\t-123.028889000000,44.930833000000");
sub ("\teasttexas_", "\teasttexas\t-95.266666666667,32.350000000000");
sub ("\teauclaire_", "\teauclaire\t-91.483333333333,44.866666666667");
sub ("\telmira_", "\telmira\t-76.900000000000,42.166666666667");
sub ("\telpaso_", "\telpaso\t-106.400000000000,31.800000000000");
sub ("\terie_", "\terie\t-80.183333333333,42.083333333333");
sub ("\teugene_", "\teugene\t-123.216666666667,44.116666666667");
sub ("\tevansville_", "\tevansville\t-87.533333333333,38.050000000000");
sub ("\tfargo_", "\tfargo\t-96.800000000000,46.900000000000");
sub ("\tfarmington_", "\tfarmington\t-90.400000000000,37.766666666667");
sub ("\tfayar_", "\tfayar\t-94.166666666667,36.000000000000");
sub ("\tfayetteville_", "\tfayetteville\t-79.016666666667,35.166666666667");
sub ("\tflagstaff_", "\tflagstaff\t-111.666666666667,35.133333333333");
sub ("\tflint_", "\tflint\t-83.733333333333,42.966666666667");
sub ("\tflorencesc_", "\tflorencesc\t-79.716666666667,34.183333333333");
sub ("\tfortcollins_", "\tfortcollins\t-105.083333333333,40.750000000000");
sub ("\tfortmyers_", "\tfortmyers\t-81.866666666667,26.583333333333");
sub ("\tfortsmith_", "\tfortsmith\t-94.366666666667,35.333333333333");
sub ("\tfortwayne_", "\tfortwayne\t-85.200000000000,41.000000000000");
sub ("\tfredericksburg_", "\tfredericksburg\t-77.466666666667,38.300000000000");
sub ("\tfresno_", "\tfresno\t-119.716666666667,36.766666666667");
sub ("\tgadsden_", "\tgadsden\t-86.000000000000,34.016666666667");
sub ("\tgainesville_", "\tgainesville\t-82.266666666667,29.683333333333");
sub ("\tgalveston_", "\tgalveston\t-94.800000000000,29.300000000000");
sub ("\tgoldcountry_", "\tgoldcountry\t-120.803056000000,38.729767000000");
sub ("\tgrandisland_", "\tgrandisland\t-98.316666666667,40.983333333333");
sub ("\tgrandrapids_", "\tgrandrapids\t-85.516666666667,42.883333333333");
sub ("\tgreenbay_", "\tgreenbay\t-88.133333333333,44.483333333333");
sub ("\tgreensboro_", "\tgreensboro\t-79.950000000000,36.083333333333");
sub ("\tgreenville_", "\tgreenville\t-82.216666666667,34.900000000000");
sub ("\tgulfport_", "\tgulfport\t-89.076111000000,30.401667000000");
sub ("\tharrisburg_", "\tharrisburg\t-76.766666666667,40.200000000000");
sub ("\tharrisonburg_", "\tharrisonburg\t-78.900000000000,38.450000000000");
sub ("\thartford_", "\thartford\t-72.650000000000,41.733333333333");
sub ("\thattiesburg_", "\thattiesburg\t-89.250000000000,31.266666666667");
sub ("\thickory_", "\thickory\t-81.383333333333,35.750000000000");
sub ("\thiltonhead_", "\thiltonhead\t-80.742947000000,32.178828000000");
sub ("\thonolulu_", "\thonolulu\t-157.916666666667,21.333333333333");
sub ("\thouston_", "\thouston\t-95.366666666667,29.983333333333");
sub ("\thudsonvalley_", "\thudsonvalley\t-74.122222000000,41.826944000000");
sub ("\thumboldt_", "\thumboldt\t-124.162778000000,40.790000000000");
sub ("\thuntington_", "\thuntington\t-82.500000000000,38.416666666667");
sub ("\thuntsville_", "\thuntsville\t-86.583333333333,34.700000000000");
sub ("\timperial_", "\timperial\t-115.558056000000,32.788056000000");
sub ("\tindianapolis_", "\tindianapolis\t-86.283333333333,39.733333333333");
sub ("\tinlandempire_", "\tinlandempire\t-117.293056000000,34.129444000000");
sub ("\tiowacity_", "\tiowacity\t-91.550000000000,41.633333333333");
sub ("\tithaca_", "\tithaca\t-76.483333333333,42.450000000000");
sub ("\tjackson_", "\tjackson\t-90.083333333333,32.316666666667");
sub ("\tjacksontn_", "\tjacksontn\t-88.916666666667,35.600000000000");
sub ("\tjacksonville_", "\tjacksonville\t-81.700000000000,30.500000000000");
sub ("\tjanesville_", "\tjanesville\t-89.031667000000,42.508330000000");
sub ("\tjerseyshore_", "\tjerseyshore\t-74.077500000000,39.927222000000");
sub ("\tjonesboro_", "\tjonesboro\t-90.700000000000,35.833333333333");
sub ("\tjoplin_", "\tjoplin\t-94.500000000000,37.150000000000");
sub ("\tjxn_", "\tjxn\t-84.466666666667,42.266666666667");
sub ("\tkalamazoo_", "\tkalamazoo\t-85.600000000000,42.283333333333");
sub ("\tkansascity_", "\tkansascity\t-94.583333333333,39.116666666667");
sub ("\tkeys_", "\tkeys\t-81.544170000000,24.666940000000");
sub ("\tkilleen_", "\tkilleen\t-97.683333333333,31.083333333333");
sub ("\tknoxville_", "\tknoxville\t-83.983333333333,35.816666666667");
sub ("\tkpr_", "\tkpr\t-119.159167000000,46.203611000000");
sub ("\tksu_", "\tksu\t-96.766666666667,39.050000000000");
sub ("\tlacrosse_", "\tlacrosse\t-91.250000000000,43.866666666667");
sub ("\tlafayette_", "\tlafayette\t-92.000000000000,30.200000000000");
sub ("\tlakecharles_", "\tlakecharles\t-93.216666666667,30.116666666667");
sub ("\tlakeland_", "\tlakeland\t-81.950000000000,28.033333333333");
sub ("\tlancaster_", "\tlancaster\t-76.300000000000,40.116666666667");
sub ("\tlansing_", "\tlansing\t-84.600000000000,42.783333333333");
sub ("\tlaredo_", "\tlaredo\t-99.450000000000,27.533333333333");
sub ("\tlascruces_", "\tlascruces\t-106.916666666667,32.300000000000");
sub ("\tlasvegas_", "\tlasvegas\t-115.166666666667,36.083333333333");
sub ("\tlawrence_", "\tlawrence\t-71.166666666667,42.700000000000");
sub ("\tlawton_", "\tlawton\t-98.416666666667,34.566666666667");
sub ("\tlexington_", "\tlexington\t-84.600000000000,38.033333333333");
sub ("\tlimaohio_", "\tlimaohio\t-84.033333333333,40.700000000000");
sub ("\tlincoln_", "\tlincoln\t-96.750000000000,40.850000000000");
sub ("\tlittlerock_", "\tlittlerock\t-92.233333333333,34.733333333333");
sub ("\tlogan_", "\tlogan\t-111.816666666667,41.750000000000");
sub ("\tlongisland_", "\tlongisland\t-73.300000000000,40.800000000000");
sub ("\tlosangeles_", "\tlosangeles\t-118.233333333333,34.050000000000");
sub ("\tlouisville_", "\tlouisville\t-85.733333333333,38.183333333333");
sub ("\tlubbock_", "\tlubbock\t-101.816666666667,33.650000000000");
sub ("\tlynchburg_", "\tlynchburg\t-79.200000000000,37.333333333333");
sub ("\tmacon_", "\tmacon\t-83.650000000000,32.700000000000");
sub ("\tmadison_", "\tmadison\t-89.333333333333,43.133333333333");
sub ("\tmaine_", "\tmaine\t-93.983333333333,44.150000000000");
sub ("\tmankato_", "\tmankato\t-93.983333333333,44.150000000000");
sub ("\tmansfield_", "\tmansfield\t-82.516666666667,40.816666666667");
sub ("\tmartinsburg_", "\tmartinsburg\t-77.983333333333,39.400000000000");
sub ("\tmcallen_", "\tmcallen\t-98.216666666667,26.200000000000");
sub ("\tmedford_", "\tmedford\t-122.866666666667,42.366666666667");
sub ("\tmemphis_", "\tmemphis\t-90.000000000000,35.050000000000");
sub ("\tmendocino_", "\tmendocino\t-123.796111000000,39.308056000000");
sub ("\tmerced_", "\tmerced\t-120.566666666667,37.383333333333");
sub ("\tmiami_", "\tmiami\t-80.266666666667,25.800000000000");
sub ("\tmilwaukee_", "\tmilwaukee\t-87.900000000000,42.950000000000");
sub ("\tminneapolis_", "\tminneapolis\t-98.066666666667,32.783333333333");
sub ("\tmobile_", "\tmobile\t-88.250000000000,30.683333333333");
sub ("\tmodesto_", "\tmodesto\t-121.000000000000,37.650000000000");
sub ("\tmohave_", "\tmohave\t-114.025833000000,35.208333000000");
sub ("\tmonroe_", "\tmonroe\t-92.033333333333,32.516666666667");
sub ("\tmontana_", "\tmontana\t-110.000000000000,47.000000000000");
sub ("\tmonterey_", "\tmonterey\t-121.900000000000,36.600000000000");
sub ("\tmontgomery_", "\tmontgomery\t-86.366666666667,32.383333333333");
sub ("\tmorgantown_", "\tmorgantown\t-79.916666666667,39.650000000000");
sub ("\tmuncie_", "\tmuncie\t-85.350000000000,40.183333333333");
sub ("\tmuskegon_", "\tmuskegon\t-86.233333333333,43.166666666667");
sub ("\tmyrtlebeach_", "\tmyrtlebeach\t-78.883653000000,33.709899400000");
sub ("\tnashville_", "\tnashville\t-86.683333333333,36.116666666667");
sub ("\tnd_", "\tnd\t-100.500000000000,47.500000000000");
sub ("\tnewhaven_", "\tnewhaven\t-73.916666666667,41.316666666667");
sub ("\tnewjersey_", "\tnewjersey\t-74.500000000000,40.000000000000");
sub ("\tnewlondon_", "\tnewlondon\t-72.100000000000,41.350000000000");
sub ("\tneworleans_", "\tneworleans\t-90.250000000000,29.983333333333");
sub ("\tnewyork_", "\tnewyork\t-74.000000000000,40.710000000000");
sub ("\tnh_", "\tnh\t-71.500000000000,44.000000000000");
sub ("\tnmi_", "\tnmi\t-86.022222000000,44.665556000000");
sub ("\tnorfolk_", "\tnorfolk\t-76.200000000000,36.900000000000");
sub ("\tnorthmiss_", "\tnorthmiss\t-88.820000000000,33.462500000000");
sub ("\tnwct_", "\tnwct\t-73.240000000000,41.790000000000");
sub ("\tocala_", "\tocala\t-82.133333333333,29.183333333333");
sub ("\todessa_", "\todessa\t-102.365556,31.863333");
sub ("\togden_", "\togden\t-112.016666666667,41.200000000000");
sub ("\toklahomacity_", "\toklahomacity\t-97.600000000000,35.400000000000");
sub ("\tolympic_", "\tolympic\t-122.900000000000,46.966666666667");
sub ("\tomaha_", "\tomaha\t-95.900000000000,41.300000000000");
sub ("\torangecounty_", "\torangecounty\t-117.620278000000,33.437778000000");
sub ("\toregoncoast_", "\toregoncoast\t-124.011111000000,44.966667000000");
sub ("\torlando_", "\torlando\t-81.383333333333,28.550000000000");
sub ("\touterbanks_", "\touterbanks\t-76.225000000000,36.295556000000");
sub ("\tpalmsprings_", "\tpalmsprings\t-116.533333333333,33.816666666667");
sub ("\tpanamacity_", "\tpanamacity\t-85.583333333333,30.066666666667");
sub ("\tparkersburg_", "\tparkersburg\t-81.566666666667,39.266666666667");
sub ("\tpennstate_", "\tpennstate\t-77.858611000000,40.791389000000");
sub ("\tpensacola_", "\tpensacola\t-87.216666666667,30.416666666667");
sub ("\tpeoria_", "\tpeoria\t-89.683333333333,40.666666666667");
sub ("\tphiladelphia_", "\tphiladelphia\t-75.250000000000,39.883333333333");
sub ("\tphoenix_", "\tphoenix\t-112.016666666667,33.433333333333");
sub ("\tpittsburgh_", "\tpittsburgh\t-80.216666666667,40.500000000000");
sub ("\tplattsburgh_", "\tplattsburgh\t-73.466666666667,44.650000000000");
sub ("\tpoconos_", "\tpoconos\t-75.359444000000,41.123056000000");
sub ("\tporthuron_", "\tporthuron\t-82.416666666667,42.983333333333");
sub ("\tportland_", "\tportland\t-122.666666666667,45.533333333333");
sub ("\tprescott_", "\tprescott\t-112.433333333333,34.650000000000");
sub ("\tprovidence_", "\tprovidence\t-71.433333333333,41.733333333333");
sub ("\tprovo_", "\tprovo\t-111.716666666667,40.216666666667");
sub ("\tpueblo_", "\tpueblo\t-104.483333333333,38.300000000000");
sub ("\tpullman_", "\tpullman\t-117.171667000000,46.732500000000");
sub ("\tquadcities_", "\tquadcities\t-90.533000000000,41.517000000000");
sub ("\tracine_", "\tracine\t-87.850000000000,42.716666666667");
sub ("\traleigh_", "\traleigh\t-78.783333333333,35.866666666667");
sub ("\treading_", "\treading\t-75.633333333333,40.333333333333");
sub ("\tredding_", "\tredding\t-122.300000000000,40.516666666667");
sub ("\treno_", "\treno\t-119.783333333333,39.500000000000");
sub ("\trichmond_", "\trichmond\t-77.333333333333,37.500000000000");
sub ("\trmn_", "\trmn\t-92.462950000000,44.023400000000");
sub ("\troanoke_", "\troanoke\t-79.966666666667,37.316666666667");
sub ("\trochester_", "\trochester\t-77.666666666667,43.116666666667");
sub ("\trockford_", "\trockford\t-89.050000000000,42.350000000000");
sub ("\trockies_", "\trockies\t-106.445358000000,39.117750000000");
sub ("\troseburg_", "\troseburg\t-123.366666666667,43.233333333333");
sub ("\troswell_", "\troswell\t-104.533333333333,33.300000000000");
sub ("\tsacramento_", "\tsacramento\t-121.500000000000,38.516666666667");
sub ("\tsaginaw_", "\tsaginaw\t-84.083333333333,43.533333333333");
sub ("\tsalem_", "\tsalem\t-123.016666666667,44.916666666667");
sub ("\tsaltlakecity_", "\tsaltlakecity\t-111.966666666667,40.766666666667");
sub ("\tsanantonio_", "\tsanantonio\t-98.466666666667,29.533333333333");
sub ("\tsandiego_", "\tsandiego\t-117.166666666667,32.733333333333");
sub ("\tsandusky_", "\tsandusky\t-82.709167000000,41.446667000000");
sub ("\tsanmarcos_", "\tsanmarcos\t-97.938829000000,29.879387000000");
sub ("\tsantabarbara_", "\tsantabarbara\t-119.833333333333,34.433333333333");
sub ("\tsantafe_", "\tsantafe\t-106.083333333333,35.616666666667");
sub ("\tsarasota_", "\tsarasota\t-82.550000000000,27.383333333333");
sub ("\tsavannah_", "\tsavannah\t-81.200000000000,32.133333333333");
sub ("\tscranton_", "\tscranton\t-75.733333333333,41.333333333333");
sub ("\tsd_", "\tsd\t-100.000000000000,44.500000000000");
sub ("\tseattle_", "\tseattle\t-122.300000000000,47.650000000000");
sub ("\tsemo_", "\tsemo\t-93.255000000000,36.637778000000");
sub ("\tsfbay_", "\tsfbay\t-77.858740000000,40.791261000000");
sub ("\tsheboygan_", "\tsheboygan\t-87.716666666667,43.750000000000");
sub ("\tshoals_", "\tshoals\t-87.650278000000,34.750833000000");
sub ("\tshreveport_", "\tshreveport\t-93.816666666667,32.466666666667");
sub ("\tsierravista_", "\tsierravista\t-110.276389000000,31.545556000000");
sub ("\tsiouxcity_", "\tsiouxcity\t-96.383333333333,42.400000000000");
sub ("\tskagit_", "\tskagit\t-122.326111000000,48.420000000000");
sub ("\tslo_", "\tslo\t-120.663192000000,35.274305000000");
sub ("\tsmd_", "\tsmd\t-76.898333000000,38.646111000000");
sub ("\tsouthbend_", "\tsouthbend\t-86.255157000000,41.672597000000");
sub ("\tsouthcoast_", "\tsouthcoast\t-71.066666666667,41.900000000000");
sub ("\tsouthjersey_", "\tsouthjersey\t-74.433333333333,39.383333333333");
sub ("\tspacecoast_", "\tspacecoast\t-80.566666666667,28.483333333333");
sub ("\tspokane_", "\tspokane\t-117.516666666667,47.633333333333");
sub ("\tspringfield_", "\tspringfieldil\t-89.666666666667,39.833333333333");
sub ("\tspringfieldil_", "\tspringfield\t-93.383333333333,37.233333333333");
sub ("\tstaugustine_", "\tstaugustine\t-81.333333333333,29.966666666667");
sub ("\tstcloud_", "\tstcloud\t-94.183333333333,45.583333333333");
sub ("\tstgeorge_", "\tstgeorge\t-113.516666666667,37.033333333333");
sub ("\tstillwater_", "\tstillwater\t-97.083333333333,36.166666666667");
sub ("\tstlouis_", "\tstlouis\t-90.383333333333,38.750000000000");
sub ("\tstockton_", "\tstockton\t-121.250000000000,37.900000000000");
sub ("\tswmi_", "\tswmi\t-83.264722000000,42.824722000000");
sub ("\tsyracuse_", "\tsyracuse\t-76.116666666667,43.116666666667");
sub ("\ttallahassee_", "\ttallahassee\t-84.366666666667,30.383333333333");
sub ("\ttampa_", "\ttampa\t-82.533333333333,27.966666666667");
sub ("\tterrahaute_", "\tterrahaute\t-87.300000000000,39.450000000000");
sub ("\ttexarkana_", "\ttexarkana\t-93.983333333333,33.450000000000");
sub ("\ttippecanoe_", "\ttippecanoe\t-86.083333333333,40.033333333333");
sub ("\ttoledo_", "\ttoledo\t-83.800000000000,41.600000000000");
sub ("\ttopeka_", "\ttopeka\t-95.633333333333,39.066666666667");
sub ("\ttreasure_", "\ttreasure\t-80.350000000000,27.466666666667");
sub ("\ttricities_", "\ttricities\t-82.400000000000,36.483333333333");
sub ("\ttucson_", "\ttucson\t-110.933333333333,32.116666666667");
sub ("\ttulsa_", "\ttulsa\t-95.900000000000,36.200000000000");
sub ("\ttuscaloosa_", "\ttuscaloosa\t-87.616666666667,33.216666666667");
sub ("\ttwinfalls_", "\ttwinfalls\t-114.483333333333,42.483333333333");
sub ("\tup_", "\tup\t-84.560300000000,43.350400000000");
sub ("\tutica_", "\tutica\t-75.383333333333,43.150000000000");
sub ("\tvaldosta_", "\tvaldosta\t-83.200000000000,30.966666666667");
sub ("\tventura_", "\tventura\t-119.183333333333,34.200000000000");
sub ("\tvictoriatx_", "\tvictoriatx\t-96.916666666667,28.850000000000");
sub ("\tvisalia_", "\tvisalia\t-119.300000000000,36.333333333333");
sub ("\twaco_", "\twaco\t-97.216666666667,31.616666666667");
sub ("\twashingtondc_", "\twashingtondc\t-77.033333333333,38.850000000000");
sub ("\twaterloo_", "\twaterloo\t-92.400000000000,42.550000000000");
sub ("\twatertown_", "\twatertown\t-76.016666666667,43.983333333333");
sub ("\twausau_", "\twausau\t-89.616666666667,44.916666666667");
sub ("\twenatchee_", "\twenatchee\t-120.316666666667,47.416666666667");
sub ("\twesternmass_", "\twesternmass\t-89.681200000000,32.767300000000");
sub ("\twestky_", "\twestky\t-84.651400000000,37.669000000000");
sub ("\twestmd_", "\twestmd\t-76.790200000000,39.072400000000");
sub ("\twestslope_", "\twestslope\t-107.880000000000,37.275277777778");
sub ("\twheeling_", "\twheeling\t-80.700000000000,40.116666666667");
sub ("\twichita_", "\twichitafalls\t-98.483333333333,33.966666666667");
sub ("\twichitafalls_", "\twichita\t-97.416666666667,37.650000000000");
sub ("\twilliamsport_", "\twilliamsport\t-76.916666666667,41.250000000000");
sub ("\twilmington_", "\twilmington\t-77.916666666667,34.266666666667");
sub ("\twinstonsalem_", "\twinstonsalem\t-80.216666666667,36.133333333333");
sub ("\tworcester_", "\tworcester\t-71.866666666667,42.266666666667");
sub ("\twv_", "\twv\t-80.969600000000,38.468000000000");
sub ("\twyoming_", "\twyoming\t-107.208500000000,42.747500000000");
sub ("\tyakima_", "\tyakima\t-120.533333333333,46.566666666667");
sub ("\tyork_", "\tyork\t-76.750000000000,39.916666666667");
sub ("\tyoungstown_", "\tyoungstown\t-80.666666666667,41.266666666667");
sub ("\tyubasutter_", "\tyubasutter\t-121.626111000000,39.134722000000");
sub ("\tyuma_", "\tyuma\t-114.616666666667,32.650000000000"); print}' > dbout/dbout.raw
echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" > dbout/dbout.kml
echo "<kml xmlns=\"http://earth.google.com/kml/2.2\">" >> dbout/dbout.kml
echo "  <Document>" >> dbout/dbout.kml
echo "    <name>Tracker</name>" >> dbout/dbout.kml
echo "    <description></description>" >> dbout/dbout.kml
echo "    <Style id=\"WHITE\">" >> dbout/dbout.kml
echo "      <LineStyle>" >> dbout/dbout.kml
echo "        <color>ffffffff</color>" >> dbout/dbout.kml
echo "        <colorMode>random</colorMode>" >> dbout/dbout.kml
echo "        <width>2</width>" >> dbout/dbout.kml
echo "      </LineStyle>" >> dbout/dbout.kml
echo "      <LabelStyle>" >> dbout/dbout.kml
echo "        <color>ff0000cc</color>" >> dbout/dbout.kml
echo "        <colorMode>random</colorMode>" >> dbout/dbout.kml
echo "        <scale>0.75</scale>" >> dbout/dbout.kml
echo "      </LabelStyle>" >> dbout/dbout.kml
echo "      <PolyStyle>" >> dbout/dbout.kml
echo "       <color>ffffffff</color>" >> dbout/dbout.kml
echo "      </PolyStyle>" >> dbout/dbout.kml
echo "    </Style>" >> dbout/dbout.kml
awk '$3 != prev { print; prev = $3 }' dbout/dbout.raw | awk '
      {print "     <Placemark>";
       print "       <name>"$1", "$2", "$3"</name>";
       print "       <styleUrl>#WHITE</styleUrl>";
       print "       <Point>";
       print "        <altitudeMode>clampToGround</altitudeMode>";
       print "        <coordinates>";
       print "          "$4",2500";
       print "         </coordinates>";
       print "       </Point>";
       print "     </Placemark>"}
' | awk '/<Placemark>/{n++}{print >"dbout/tmp/path"n".tmp" }'
awk '{print $1}' dbout/dbout.raw | sort | uniq | awk '{print "echo -n \"\" > dbout/tmp/"$1}' > crtnum.sh
find dbout/tmp/ -mindepth 1 | xargs grep -H '<name>' | column -t -s">"","":" | awk '{print "cat "$1" >> dbout/tmp/"$3}' > tmpmv.sh
sh crtnum.sh
sh tmpmv.sh
awk '$3 != prev { print; prev = $3 }' dbout/dbout.raw | awk '{print $1"\t"$4}' | awk '
$1!=p1{print "         </coordinates>";
       print "       </LineString>";
       print "     </Placemark>";
       print "    </Folder>";
       print "    <Folder>";
       print "     <name>"$1"</name>";
       system("cat dbout/tmp/"$1)
       print "     <Placemark>";
       print "       <name>"$1"</name>";
       print "       <styleUrl>#WHITE</styleUrl>";
       print "       <LineString>";
       print "        <extrude>1</extrude>";
       print "        <tessellate>1</tessellate>";
       print "        <altitudeMode>clampToGround</altitudeMode>";
       print "        <coordinates>"}
$2!=p2{print "          "$2",2500";p1=$1}' | awk '{if (NR>4) {print}}' >> dbout/dbout.kml
echo "         </coordinates>" >> dbout/dbout.kml
echo "       </LineString>" >> dbout/dbout.kml
echo "     </Placemark>" >> dbout/dbout.kml
echo "    </Folder>" >> dbout/dbout.kml
echo "  </Document>" >> dbout/dbout.kml
echo -n "</kml>" >> dbout/dbout.kml


Last edited by otheus; 02-03-2009 at 05:33 AM.. Reason: code tags, lightbulb icon, closed thread
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to block first bash script until second bash script script launches web server/site?

I'm new to utilities like socat and netcat and I'm not clear if they will do what I need. I have a "compileDeployStartWebServer.sh" script and a "StartBrowser.sh" script that are started by emacs/elisp at the same time in two different processes. I'm using Cygwin bash on Windows 10. My... (3 Replies)
Discussion started by: siegfried
3 Replies

2. Homework & Coursework Questions

Small project help

hey..i need help with one small project..basically at the shell script level..have for that approx 2 days..and willing to pay..actually before went to fiverr.com and www.enotes.com/homework-help and www.upwork.com and yourhomeworkhelp.org but they are charging me a lot for this type of... (1 Reply)
Discussion started by: steph33
1 Replies

3. Shell Programming and Scripting

Assistiance in small bash script

Hi All, i wrote an simple script, that will monitor some services ,and if particular service will found down ,the code will restarts the service and message about the process will be sent on mail. I have a problem somewhere in the syntax that the script initializes the service even if is up... (6 Replies)
Discussion started by: Aviel.shani
6 Replies

4. Shell Programming and Scripting

Making a bash script and small C program for a homework assignment

Here's the assignment. I'll bold the parts that are rough for me. Unfortunately, that's quite a bit lol. The syntax is, of course, where my issues lie, for the most part. I don't have a lot of programming experience at all :/. I'd post what I've already done, but I'm so lost I really don't know... (1 Reply)
Discussion started by: twk101
1 Replies

5. Programming

Need Help in Small Project

I need help with a small project that I am doing for my client. My clients have a company that does staffing for hotel workers and when they sign up with him, they have to watch 4 separate training videos. What I need help with is a script of some sort that will pop-up a sign in box of some sort... (2 Replies)
Discussion started by: AimyThomas
2 Replies

6. UNIX for Dummies Questions & Answers

need help with small script

Hi I have the below file Name=abc Yr=2011 Mon=05 My script should be in such a way that whenever I run it then Month should increment by 1 and if the month is 12 then when I run the script then year should incremented by 1 and month Should become 01(I.e jan) Thanks for the help in... (6 Replies)
Discussion started by: kishu
6 Replies

7. Programming

Project ideas for GNU/Linux C and BASH development

Hello everyone... I'm trying to find an interesting project to work on for my master thesis. I like GNU/Linux C development and BASH scripting. Please give me any idea that flashes in your mind. I thank you in advance... (3 Replies)
Discussion started by: jonx
3 Replies

8. Shell Programming and Scripting

bash script working for small size files but not for big size files.

Hi, I have one file stat. Stat file contents are as follows: for example. H50768020040913,00260100,507680,13,0000000643,0000000643,00000,0000 H50769520040808,00260100,507695,13,0000000000,0000000000,00000,0000 H50770620040611,00260100,507706,13,0000000000,0000000000,00000,0000 Now i... (1 Reply)
Discussion started by: davidpreml
1 Replies

9. UNIX for Dummies Questions & Answers

Learning to script...want a project to work on (making a small MUD)

Will it be possible to make a MUD using unix script(bash)? I know I can easilly make a "dungeon" with different rooms I can run through and such(kinda like a maze game) . What I did was made a differnt shell script for each room and used CASE for the different places to go, ex. when you press N for... (2 Replies)
Discussion started by: learningtoscrip
2 Replies
Login or Register to Ask a Question