Help with Data Positioning from Columns in a flat file.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with Data Positioning from Columns in a flat file.
# 1  
Old 06-14-2006
Help with Data Positioning from Columns in a flat file.

Hi All,
I have used this forum many times to solve my many scripting problems. This time, I would like to seek some answers to a problem that I've been head scratching quite a bit on.

My Example:
I am converting a 2000-byte file into a 300-byte file
this file has no delimiters and hardly any spaces inbetween the cols...
I have successfully converted most of the 2000-byte cols into the 300-byte file format by using awk's substr function, but I am having an issue in getting 1 col of the 2000-byte file to space properly in the 300-byte file.
SUBSTR would work perfectly if the data was positioned all the same way.

My dilema,
The 2000-byte column starts at position 150 and ends at 180 (30 bytes)
this col contains city,state,zip code info
and the positioning of these items varies per record...
example record#1 anytown us 11111
example record#2 any town us 11111
example record#3 anytown us11111
etc,

The requirements,
I have to specifically place the city info into position 159-181 of the 300-byte file
I have to specifically place the state info into pos 182-183
& I have to spcifically place the zip info into pos 184-188

Not sure what would work best, since I have to keep intact the integrity of each record as well.....and then manipulate this col .... without trashing anything else....I would appreciate any ideas...
thanks...
Smilie
# 2  
Old 06-14-2006
1. do all of your 'columns' starts at the same position?
2. do all the columns are of the form
Code:
<town name> US 5numberZIP

3. Is the string US always the same OR is it the 2-letter state abbreviation?
4. a couple of sample records/lines would help to see the 'pattern'
# 3  
Old 06-14-2006
Hi...vergsh99

1) yes it all starts at the first position, BUT the data is not specific after that...tho they do follow this pattern.. city then state then zip

2) yes its that format...but they all have various positioning

3) US was just an example...it can be any state abbr....TX, CA, etc..

4) examples:

BIG COVE TANNRY PA 17212
WEST CHESTER PA 19382
PEARL RIVER NY10965
JAMAICA NY 11432


I need it look like this
Pos 159-181 Pos 182-188

BIG COVE TANNRY PA17212
WEST CHESTER PA19382
PEARL RIVER NY10965
JAMAICA NY11432
# 4  
Old 06-14-2006
here's an idea to start with.......

assuming your 'column' starts at position '10' - you can change the code to reflect the 'reality'.

nawk -f oo.awk oo.txt

oo.txt:
Code:
a;skldfj BIG COVE TANNRY PA17212 sakdlfjaslkdf
zxm,cvn  WEST CHESTER PA19382 a;sldkfja;skldj asdlk
a;skldfj PEARL RIVER NY10965 127 asdkfj asdlfkj
wut sdf  JAMAICA NY11432  sd sdlkfj 0938

oo.awk:
Code:
BEGIN {
  FLRstart=10
  PAT=".*[A-Z][A-Z].*[0-9][0-9][0-9][0-9][0-9]"
  PATstate="[A-Z][A-Z][ ]*[0-9][0-9][0-9][0-9][0-9]$"
}

{
   _sub=substr($0, FLRstart)
   #printf("_sub->[%s]\n", _sub)

   if( match(_sub, PAT) )
     #printf("subSTR->[%s]\n", substr(_sub, RSTART, RLENGTH))

   _column=substr(_sub, 1, RSTART+RLENGTH-1)
   printf("_column->[%s]\n", _column)

   end=substr(_sub, RSTART)

   zip=substr(_column, RLENGTH-4)

   if (match(_column, PATstate) )
     state=substr(_column, RSTART, 2)

   town=substr(_column, 1, RSTART-2)

   printf("town->[%s]\n", town)
   printf("state->[%s]\n", state)
   printf("zip->[%s]\n", zip)

}

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Oracle table extract: all columns are not converting into pipe delimited in flat file

Hi All, I am writing a shell script to extract oracle table into a pipe dilemited flat file. Below is my code and I have attached two files that I have abled to generate so far. 1. Table.txt ==> database extract file 2. flat.txt ==> pipe delimited after some manipulation of the original db... (5 Replies)
Discussion started by: express14
5 Replies

2. Shell Programming and Scripting

Getting data from a flat file based on condition

Hi, I have a flaty file from which i am fetching few columns in tablular form as below code. Now i want to fetch the column 6 and 7 in below code only if it either of them is non zero.However below startement awk -F, '$6==0 && $7==0{exit 1}' ${IFILE} is not working..Not sure where is the... (36 Replies)
Discussion started by: Vivekit82
36 Replies

3. Shell Programming and Scripting

Insert empty columns in a flat file

Hi, I have a tab delimited flat file, for example shown below Name Desg Loc a b c d e fI want to insert an empty column inbetween the column Desc and Loc, the result should be like shown below: Name LName Desg Loc a b c d e ... (6 Replies)
Discussion started by: sampoorna
6 Replies

4. Shell Programming and Scripting

Read flat file upto certain number of columns

Hello Guys Please help me with the below issue I want to read a flat file source upto certain number of columns Say my flat file has 30 columns but I want to read upto 25 columns only How come the above issue can be addressed? Thanks a lot!!!! (1 Reply)
Discussion started by: Pratik4891
1 Replies

5. UNIX for Dummies Questions & Answers

Audit Flat File - # of Columns / Rows

We receive a file which usually has 40 to 50 million rows. I want to set up an audit process by which everytime we receive a file we audit it for # of rows and total number of columns. So if the # of rows is around 1 million on a particular day, I want to raise a flag or send an email....and if... (3 Replies)
Discussion started by: priya33184
3 Replies

6. Shell Programming and Scripting

To read a flat file containing XML data

I have a file something like this:aaaa.xml content of the file is 0,<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <storeInformation xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <s> <BRANCH_NO>3061</BRANCH_NO> <BRANCH_NAME>GREEN EXPRESS</BRANCH_NAME> ... (4 Replies)
Discussion started by: kmanivan82
4 Replies

7. UNIX for Dummies Questions & Answers

Normalize Data and write to a flat file

All, Can anyone please help me with the below scenario. I have a Flat file of the below format. ID|Name|Level|Type|Zip|MAD|Risk|Band|Salesl|Dealer|CID|AType|CValue|LV|HV|DCode|TR|DU|NStartDate|UserRole|WFlag|EOption|PName|NActivationDate|Os|Orig|Cus|OType|ORequired|DType 03|... (10 Replies)
Discussion started by: sp999
10 Replies

8. Shell Programming and Scripting

Load data from a flat file to oracle.

I have a flat file with records like Header 123 James Williams Finance2000 124 Pete Pete HR 1500 125 PatrickHeather Engg 3000 Footer The structure is: Eno:4 characters Name:8 characters Surname : 9 characters Dept:7 characters Sal:4characters These are sample... (1 Reply)
Discussion started by: Shivdatta
1 Replies

9. Shell Programming and Scripting

inserting data into a table from a flat file

Hi, I want to insert data into a table from a flat file, the file is having around 25 columns and some 10,000 records. The columns values are seperated by a space. Thanks (1 Reply)
Discussion started by: ss_ss
1 Replies
Login or Register to Ask a Question