Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Validating fixed length field...

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Validating fixed length field...
# 1  
Old 05-08-2003
Validating fixed length field...
What's wrong with this part of my script?
I just want to put each column from a fixed length file into a variable so I can validate each field later in the script.
Code:
exec< myfile.dat
while read afile; do
   a=`echo $(echo $afile |cut -c1-10)`
   echo "$a"
   b=`echo $(echo $afile |cut -c11-20)`
   echo "$b"
   c=`echo $(echo $afile |cut -c21-22)`
   echo "$c"
   d=`echo $(echo $afile |cut -c23-40)`
   echo "$d"
   e=`echo $(echo $afile |cut -c41-41)`
   echo "$e"
   f=`echo $(echo $afile |cut -c42-48)`
   echo "$f"
   g=`echo $(echo $afile |cut -c49-52)`
   echo "$g"
   rtecd=`echo $(echo $afile |cut -c53-60)`
   echo "$h"
   i=`echo $(echo $afile |cut -c61-110)`
   echo "$i"
done

I put the echo in so that it would see each record and column as the record is being processed.
When I run it, my script echos out only up to 'g' for each record and nothing else. Which means I can't validate all of the fields.

I'm not sure what I'm doing wrong. I saw this syntax from another posting but it doesn't seem to work for me. There's no 'delimiter' for me to work with so I'm not sure how to tackle. Is there a better way?? You guys have helped me alot with all this stuff in the past so thanks in advance.

Gianni

added code tags for readability --oombera
Last edited by oombera; 02-20-2004 at 02:54 AM..
# 2  
Old 05-08-2003
Well it's not echoing $h because you're not storing anything in $h.. you have to change rtecd=`... to h=`...

Here's an alternative way, but code-wise, it's just about as long:
Code:
while read LINE
do
 a=`echo $LINE | awk '{print substr ($0, 1, 10)}'`
 b=`echo $LINE | awk '{print substr ($0, 11, 10)}'`
 c=`echo $LINE | awk '{print substr ($0, 21, 2)}'`
 d=`echo $LINE | awk '{print substr ($0, 23, 18)}'`
 e=`echo $LINE | awk '{print substr ($0, 41, 1)}'`
 f=`echo $LINE | awk '{print substr ($0, 42, 7)}'`
 g=`echo $LINE | awk '{print substr ($0, 49, 4)}'`
 h=`echo $LINE | awk '{print substr ($0, 53, 8)}'`
 i=`echo $LINE | awk '{print substr ($0, 61, 40)}'`
done < parse

There's probably a quicker way; not sure what that is..
# 3  
Old 05-08-2003
Thanks for the code. That worked but it also brought me back to my initial problem (with the cut command I used). It will not print anything beyond 'g' (I corrected my typo above):

a: 1234567890
b: 2003050812
c: 45
d: av12la2456k8jk1234
d: P
f: 23kkuzj
g: abcd
h:
i:

I know that 'h' and 'i' have data in them but these do not print. Do you know if there is a limit to how many variables we can define in a read statement in ksh on AIX?

I actually have variables defined to 'n' but I'm showing you just a sample of it. I tried ulimit 99999 but that didn't do anything.

Gianni
# 4  
Old 05-08-2003
Your code works (after changing rtecd=`... to h=`... as suggested by Oombera) on both Solaris (8) ksh and HP-UX (11) ksh. I don't believe AIX would have done something different with the cut command.

Try looking at your data - test it by using a test file such as the following:

test.dat
11111111112222222222334444444444444444445666666677778888888899999999999999999999999999999999
aaaaaaaaaabbbbbbbbbbccddddddddddddddddddefffffffgggghhhhhhhhiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii iiiii

If it prints this correctly then there is something in your data file that is causing a problem.
# 5  
Old 05-08-2003
I made the correction but I it didn't help. I read somewhere that there is a limit (54 byte) that can be read at any one time, is that true? The suggestion was to set the ulimit to 99999 to bypass this limit but it didn't help (I think I saw that in one of the postings here).

The file I'm working on contains only 10 test records but the record length are 120 in length.

When I put each column into a variable, anything after a certain point, in this case, I said 'g', would be blank and will not print. So my guess is that they're not being read. I'm thinking that it might be the 54 byte limit that is preventing me from seeing those fields.

I'm not sure what is going on. It's not difficult to duplicate the awk command line above so I know I'm not messing up there.

I would be surprised by this limitation so it must be the way I'm handling variable declarations in the while loop?

Is this also the only option when you do not have a delimiter, like a pipe or space or tab because it would have been easy to use FS="|", etc?

Thanks.
# 6  
Old 05-08-2003
I realized now that when I'm reading the file, awk is ignoring all the blanks that exists in the file and is getting rid of those blanks which makes a field right or left justified prior to setting the variables.

How do I prevent awk or whatever program from ignoring the blanks and keeping the columns intact? Also, some fields are zero filled and needed to retain their format.

Thanks.
# 7  
Old 05-09-2003
Quote the variables wherever you use them...
Code:
while read LINE
do
 a=`echo "$LINE" | awk '{print substr ($0, 1, 10)}'`
 b=`echo "$LINE" | awk '{print substr ($0, 11, 10)}'`
 ...
 ...
 h=`echo "$LINE" | awk '{print substr ($0, 53, 8)}'`
 i=`echo "$LINE" | awk '{print substr ($0, 61, 40)}'`
 echo "$a"
 ...
 ...
done < fileName

Or, using your code:
Code:
...
...
a=`echo "$(echo $afile |cut -c1-10)"`
echo "$a"
b=`echo "$(echo $afile |cut -c11-20)"`
echo "$b"
c=`echo "$(echo $afile |cut -c21-22)"`
...
...

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Convert variable length record to fixed length

Hi Team, I have an issue to split the file which is having special chracter(German Char) using awk command. I have a different length records in a file. I am separating the files based on the length using awk command. The command is working fine if the record is not having any... (7 Replies)
Discussion started by: Anthuvan
7 Replies

2. Shell Programming and Scripting

Make it to fixed length

Hi Team, I have a different length records in my text file.I would like to make all the records with same length. I want to check the maximum lenth and all other records make the same length It's urgent request. Thanks in Advance (2 Replies)
Discussion started by: Anthuvan
2 Replies

3. Shell Programming and Scripting

Flat file-make field length equal to header length

Hello Everyone, I am stuck with one issue while working on abstract flat file which i have to use as input and load data to table. Input Data- ------ ------------------------ ---- ----------------- WFI001 Xxxxxx Control Work Item A Number of Records ------ ------------------------... (5 Replies)
Discussion started by: sonali.s.more
5 Replies

4. Shell Programming and Scripting

changing a variable length text to a fixed length

Hi, Can anyone help with a effective solution ? I need to change a variable length text field (between 1 - 18 characters) to a fixed length text of 18 characters with the unused portion, at the end, filled with spaces. The text field is actually field 10 of a .csv file however I could cut... (7 Replies)
Discussion started by: dc18
7 Replies

5. Shell Programming and Scripting

Make variable length record a fixed length

Very, very new to unix scripting and have a unique situation. I have a file of records that contain 3 records types: (H)eader Records (D)etail Records (T)railer Records The Detail records are 82 bytes in length which is perfect. The Header and Trailer records sometimes are 82 bytes in... (3 Replies)
Discussion started by: jclanc8
3 Replies

6. UNIX for Dummies Questions & Answers

Validating input based on fixed number of fields

Yes, i did... let me state my problem in more detail Inputs: I have one input CSV file And, i have stored no. of comma each line should in a variable. e.g. $ cat cmt.csv this, is a ,comma ,count test1 ,,this, is a ,comma ,count test2 this, is a ,comma ,count test3... (6 Replies)
Discussion started by: Dipali
6 Replies

7. Programming

parsing fixed length field with yacc/bison

How to specify the token length in a yacc file? sample input format <field1,data type ans,fixed length 6> followed by <field2,data type ans,fixed length 3> Example i/p and o/p Sample Input: "ab* d2 9o" O/p : "Field1 Field2 " yacc/bison grammar: record :... (1 Reply)
Discussion started by: sungita
1 Replies

8. UNIX for Dummies Questions & Answers

Convert a tab delimited/variable length file to fixed length file

Hi, all. I need to convert a file tab delimited/variable length file in AIX to a fixed lenght file delimited by spaces. This is the input file: 10200002<tab>US$ COM<tab>16/12/2008<tab>2,3775<tab>2,3783 19300978<tab>EURO<tab>16/12/2008<tab>3,28523<tab>3,28657 And this is the expected... (2 Replies)
Discussion started by: Everton_Silveir
2 Replies

9. UNIX for Dummies Questions & Answers

What the command to find out the record length of a fixed length file?

I want to find out the record length of a fixed length file? I forgot the command. Any body know? (9 Replies)
Discussion started by: tranq01
9 Replies

10. Shell Programming and Scripting

creating a fixed length output from a variable length input

Is there a command that sets a variable length? I have a input of a variable length field but my output for that field needs to be set to 32 char. Is there such a command? I am on a sun box running ksh Thanks (2 Replies)
Discussion started by: r1500
2 Replies
Login or Register to Ask a Question