Manupulating Records in a fixed width file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Manupulating Records in a fixed width file
# 1  
Old 07-23-2010
Data Manupulating Records in a fixed width file

I am trying to determine what would be a fast and simple way to manipulate data that comes in a fixed width format. This data has 6 segments within a record. Each record needs to written out with a header and the 6 segments. Based on the value in column #6 the fields will be defined accordingly. The value in column #6 could be a number in the range of 1 -6.

The field that separates each record is in field 221. If column 221 has an "H", this is header field and all segments 1 - 6 need to be written together.

The header line has the following properties:
There is an "H" in column 221
column 1-6 = CIU
column 7-11 = DIS
column 12-16 = ZIP
column 21-70 = NAME

This header should be written at the top of each record. Under the header should be the data, which looks like this.

If column 221 has anything other than an "H", then it is the data that goes with the previous header. The next "H" in column 221 is the start of a new record.

If we determined that we have a new record that has an "H" in the 221 column the data following it should be assigned a label based on the value of column #6. Column #6 should always be an integer between 1 - 6.

If column 6 = 1
column 7-13 = TAP
column 20-21 = COL
column 29-30 = PMF
column 31-39 = NSS
column 40-47 = DOB
column 48-50 = AGE
...
...
column 208-216 = PAT

If column 6 = 2
column 14-21 = PR1
column 22-29 = PR2
column 30-37 = PR3
...
...
column 78-79 = NUM
column 80-84 = ALL
...
column 217-220 = REF_2

If column 6 = 3
column 14-16 = DRR
column 17-18 = CDM
column 24-31 = CASE
column 39-39 = MMM
...
column 77-81 = TIN

If column 6 = 4
column 204-213 = ID
column 214-215 = LOC

If column 6 = 5
column 128-145 = MCP_DI
column 146-146 = MCP_CD
column 147-165 = PRG_DI
column 166-195 = PRG_MN

If column 6 = 6
column 94-101 = 9PR
column 102-109 = 10PR
column 110-117 = 11PR
column 118-125 = 12PR
...
column 214-217 = HOLD


Here is an example record

Z18BTT0025 234684929347293TELESD DEF ..... over to field 221 = "H"
0025 1282190052150510282205296049821900521....over to field 221 = "E"
0025 2900521505127492001810787819191115..... over to field 221 = ""
0025 30660000000000000000000000000089723... over to field 221 = ""
0025 49090214430660000000000000000000000000... over to field 221 = "
0025 56051274942731 4280 27652 V090 ... over field 221 = ""
0025 60510282 over field 191 = DOE, JOHN... over field 221 = ""
F19AYY0025 2346789123456789ZTREWQ RRR ..... over to field 221 = "H"
0025 112749200181078781919111587Y87Y... over to field 221 = "G"
0025 211112222 D012345 D012345 D023456 R13123 over to field 221 = ""
0025 300000000000000000000000000051036842... over to field 221 = ""
0025 40660000000000000000000000000... over to field 221 = ""
0025 5060825Y25YFX4967443309A... over to field 221 = ""
0025 61234567... over to field 221 = ""
H20AYY0025 2346789123456789ZTREWQ EFD ..... over to field 221 = "H"


I realize that this a tough one to follow and any help would be appreciated. I have attempted doing this with awk and substr, but I am not getting any of logic to work. I think it may be best to do this with Perl, but my Perl programming is not that great. Now I am wondering am I going down the right path at all.
# 2  
Old 07-23-2010
I believe it would be easier if you post a sample data with complete records (as an attachment) and an example of the desired output.
# 3  
Old 07-23-2010
Hi.

And, while you are posting and pondering, please peruse perldoc -f pack and perldoc -f unpack.
Quote:
Use unpack to to extract fixed-width fields.
-- Perl Best Practices, p 154 with examples
Best wishes (with apologies for alliteration addiction) ... cheers, drl
# 4  
Old 07-27-2010
Quote:
Originally Posted by radoulov
I believe it would be easier if you post a sample data with complete records (as an attachment) and an example of the desired output.


I have attached a sample file with data. Hope this helps.
# 5  
Old 07-30-2010
Sorry for the delay,
could you please also post an example of the desired output?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Answers to Frequently Asked Questions

Fixed width file issue and resolutions

I have a fixed width file with no separators , but the file has JUNK characters in it and i know how to remove it. but in few cases these junk characters having created a mess by splitting single row into 2 or 3 rows. I need to put them back to a single line like rest of the rows. This fixed... (2 Replies)
Discussion started by: santoshkm
2 Replies

2. Shell Programming and Scripting

Replace using awk on fixed width file.

All, I used to use following command to replace specific location in a fixed width file. Recently looks like my command stopped working as intended. We are on AIX unix. awk 'function repl(s,f,t,v) { return substr(s,1,f-1) sprintf("%-*s", t-f+1, v) substr(s,t+1) } NR<=10 {... (3 Replies)
Discussion started by: pinnacle
3 Replies

3. Shell Programming and Scripting

Alter Fixed Width File

Thank u so much .Its working fine as expected. ---------- Post updated at 03:41 PM ---------- Previous update was at 01:46 PM ---------- I need one more help. I have another file(fixed length) that will get negative value (ex:-00000000003000) in postion (98 - 112) then i have to... (6 Replies)
Discussion started by: vinus
6 Replies

4. UNIX for Dummies Questions & Answers

Length of a fixed width file

I have a fixed width file of length 53. when is try to get the lengh of the record of that file i get 2 different answers. awk '{print length;exit}' <File_name> The above code gives me length 50. wc -L <File_name> The above code gives me length 53. Please clarify on... (2 Replies)
Discussion started by: Amrutha24
2 Replies

5. Shell Programming and Scripting

Apply condition on fixed width file and filter records

Dear members.. I have a fixed width file. Requirement is as below:- 1. Scan each record from this fixed width file 2. Check for value under field no "6" equals to "ABC". If yes, then filter this record into the output file Please suggest a unix command to achieve this, my guess awk might... (6 Replies)
Discussion started by: sureshg_sampat
6 Replies

6. Shell Programming and Scripting

Comparing two fixed width file

Hi Guys I am checking the treads to get the answer but i am not able to get the answer for my question. I have two files. First file is a pattern file and the second file is the file i want to search in it. Output will be the lines from file2. File1: P2797f12af 44751228... (10 Replies)
Discussion started by: anshul_er
10 Replies

7. Shell Programming and Scripting

Fixed-Width file from Oracle

Hi All, I have created a script which generates FIXED-WIDTH file by executing Oracle query. SELECT RPAD(NVL(col1,CHR(9)),20)||NVL(col2,CHR(9))||NVL(col3,CHR(9) FROM XYZ It generates the data file with proper alignment. But if same file i transfer to windows server or Mainframe... (5 Replies)
Discussion started by: Amit.Sagpariya
5 Replies

8. UNIX Desktop Questions & Answers

Help with Fixed width File Parsing

I am trying to parse a Fixed width file with data as below. I am trying to assign column values from each record to variables. When I parse the data, the spaces in all coumns are dropped. I would like to retain the spaces as part of the dat stored in the variables. Any help is appreciated. I... (4 Replies)
Discussion started by: sate911
4 Replies

9. Shell Programming and Scripting

Extracting records with unique fields from a fixed width txt file

Greetings, I would like to extract records from a fixed width text file that have unique field elements. Data is structured like this: John A Smith NY Mary C Jones WA Adam J Clark PA Mary Jones WA Fieldname / start-end position Firstname 1-10... (8 Replies)
Discussion started by: sitney
8 Replies

10. UNIX for Dummies Questions & Answers

Fixed Width file using AWK

I am using the following command at the Unix prompt to make my 'infile' into a fixed width file of 100 characters. awk '{printf "%-100s\n",$0}' infile > outfile However, there are some records with a special character "©" These records are using 3 characters in place of one and my record... (2 Replies)
Discussion started by: alok.benjwal
2 Replies
Login or Register to Ask a Question