include NULLs in line length check


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting include NULLs in line length check
# 1  
Old 06-29-2009
include NULLs in line length check

Hello,
I am checking the length of each line of a fixed length file and making sure all lines are 161 length. My problem is that some files contain null characters which gets stripped out of my echo. How do I have the NULLs included in my check? (and I cannot replace or sub the NULL values with anything either).
Thanks for the help....


#!/bin/ksh
RECORD_LEN=161
RECORD_COUNT=0
while read -r line
do
ACTUAL=`echo "${#line}"`
printf "\n$line\n"
echo $ACTUAL
if [ ${RECORD_LEN} -ne $ACTUAL ]; then
exit;
fi
done < file
# 2  
Old 06-29-2009
try using awk
Code:
awk '{if(length($0)!=161){exit}else{print length($0)}}' filename

# 3  
Old 06-30-2009
mmm

Thanks - but that code actually stops at the first NULL and truncates the line length and fails the length check.
Any other ideas
# 4  
Old 06-30-2009
your original logic also do the same so i wrote exit once length is not equal to 161..
# 5  
Old 06-30-2009
my point was that both your code and mine are not working. Let me try to explain my problem again. I have a fixed length file that I am reading every line and making sure every line is exactly 161 length. Problem is that some lines contain NULL characters (or low 'MAINFRAME' values). These are not space values.

For example, lets say that my file has one record and has a NULL right in the middle at position 80. My script is returning 160 as length and throwing out the NULL altogether. Your awk code is stopping at the NULL and returning the line length as 79. I need code to return all characters including the NULL values.
I was thinking instead of trying to get the actual line length I could go to the end of each line and retrieve the line position from there. I think that would be better but cannot find relevant code for that.
Thanks
# 6  
Old 06-30-2009
Code:
wc -c datafile | read value junk
if [[  $$( $value % 161 )) -ne 0 ]] ; then
      echo "data file does not have 161 lrecl record(s)"
else
      echo "ok"
fi

If you have to check every single line try something like this
Code:
dd if=datafile of=testfile cbs=161 block
cksum datafile | read chk1 junk
cksum testfile | read chk2 junk
if [[ "$chk1" != "$chk2" ]] ; then
   echo data format error in datafile"
fi

# 7  
Old 06-30-2009
Please post a sample of how the input file looks like...
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Check for length which exceeds specified length in a line

Hi, I have a issue, I need to loop through a comma delimited file and check for the length which exceeds specified length , if Yes truncate the string. But my problem is , I do not have to check for all the fields and the field lenght is not same for all the fields. For ex: Say my line... (9 Replies)
Discussion started by: rashmisb
9 Replies

2. Shell Programming and Scripting

perl file, one line code include "length, rindex, substr", slow

Hi Everyone, # cat a.txt a;b;c;64O a;b;c;d;ee;f # cat a.pl #!/usr/bin/perl use strict; use warnings; my $tmp3 = ",,a,,b,,c,,d,,e,,f,,"; open(my $FA, "a.txt") or die "$!"; while(<$FA>) { chomp; my @tmp=split(/\;/, $_); if ( ($tmp =~ m/^(64O)/i) || ($tmp... (3 Replies)
Discussion started by: jimmy_y
3 Replies

3. Shell Programming and Scripting

Check length of record

Hi, I have a problem, please help me, I have a flat file like this: P00000000088888888999999999 0000999903 000000000000000000 P00000000077777777000000000 0000999903 000000000000000000 P00000000044444444333333333 0000999903 00000000000000000079875 P00000000066666666111111111 0000999903 ... (5 Replies)
Discussion started by: DebianJ
5 Replies

4. Shell Programming and Scripting

Deleting Characters at specific position in a line if the line is certain length

I've got a file that would have lines similar to: 12345678 x.00 xx.00 x.00 xxx.00 xx.00 xx.00 xx.00 23456781 x.00 xx.00 xx.00 xx.00 xx.00 x.00 xxx.00 xx.00 xx.00 xx.00 34567812 x.00 xx.00 x.00 xxx.00 xx.00 xx.00 xx.00 45678123 x.00 xx.00 xx.00 xx.00 xx.00 x.00 xxx.00 xx.00 xx.00 xx.00 xx.00... (10 Replies)
Discussion started by: Cailet
10 Replies

5. Shell Programming and Scripting

sort truncates line when they contain nulls

When I try to sort a file where some records contain nulls i.e. hex 00 the sort truncates the record when it reaches the null and writes message: "sort: warning: missing NEWLINE added at end of input file myfile" I'm assuming from this that the sort sees the null as a special character and... (6 Replies)
Discussion started by: ArthurWaik
6 Replies

6. Shell Programming and Scripting

How to check Null values in a file column by column if columns are Not NULLs

Hi All, I have a table with 10 columns. Some columns(2nd,4th,5th,7th,8th and 10th) are Not Null columns. I'll get a tab-delimited file and want to check col by col and generate seperate error code for each col eg:102 if 2nd col value is NULL and 104 if 4th col value is NULL so on... I am a... (7 Replies)
Discussion started by: Mandab
7 Replies

7. AIX

check length

I'm not familiar with unix. I need know how to check parameter (length) when i need check the userid. lets say my charaters is 3, how to check I must key-in 3 word before system can continue to another step echo "\n\n\n\tPlease Enter User ID: \c" read userid if then echo... (1 Reply)
Discussion started by: wanasmadi
1 Replies

8. Shell Programming and Scripting

Check length of Shell Variable

I am using a Korn Shell script.. I need to verify the length of a variable.. ie number=12345 I need then to check 12345 to make sure its no longer than 5 digits... Can you help? (4 Replies)
Discussion started by: frustrated1
4 Replies
Login or Register to Ask a Question