Verifying Record Length


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Verifying Record Length
# 8  
Old 05-11-2013
Yes, I always put the opening { on the if line.
On the other hand, a correct awk should also handle two lines ... But all the other code looks okay.

$0 is the current line.
NR is the line number.
BTW I also put this in one line:
Code:
} else {


Last edited by MadeInGermany; 05-11-2013 at 04:54 PM..
This User Gave Thanks to MadeInGermany For This Post:
# 9  
Old 05-11-2013
Quote:
Originally Posted by SoloXX
Hi Don,

Yes, sorry...yes I know it's an awk script based on the extension of the script.

==============
The process that invokes the awk script:

1. Check for source file arrival
2. Header date verified on incoming file from internal sql query to database
lookup table
3. Translate unwanted Characters to spaces
4. Move file from source directory to processing directory

After the above completes, the Verify Record Length AWK script automatically begins.

1. Veryifies total records and checks record lengths
===============

What Linux system are you using? Red Hat Enterprise Linux Server release 5.8 (Tikanga)
What shell are you using on Linux? Korn
What version of AIX are you using on your AIX server? 5.3.0.0
What shell is being used on your AIX server? Korn
Is the input data file encoded in EBCDIC? Do not know.
I repeat (from my 1st posting in this thread:
Quote:
Please show us the exact command line that is being used to invoke awk using this script file and show us how the input file is being passed to awk.
I understand the English description of the processing that is going on. To help you with your problem, we need to see the exact Korn shell commands that are being used to run your awk script. Saying:
Quote:
After the above completes, the Verify Record Length AWK script automatically begins.
doesn't give us any way to evaluate what might be going wrong.
# 10  
Old 05-11-2013
Sorry Don, but our Batch document doesn't provide us that much detail yet as it is being put together at the moment.

On anohter note, is there a command I can use in Unix that can tell me the first line's record length if I navigate to the source directory?

I tried wc -c <insert filename> but it gives me the TOTAL character counts.
# 11  
Old 05-11-2013
Length of first line:
Code:
awk 'NR==1 {print length($0)}'

If nothing else is to be processed, immediate exit is faster
Code:
awk '{print length($0); exit}'

---------- Post updated at 04:23 PM ---------- Previous update was at 04:15 PM ----------

Don suspects you run your script by the wrong interpreter. I can hardly believe this since you provided logfile. But to be sure I agree you should show how you run it.
# 12  
Old 05-12-2013
Hi SoloXX,
When I last posted on this thread, I hadn't seen the sample input that you posted in message #3 in this thread 1 or 2 minutes before I hit the Submit Reply button. Now that we know that you have no idea what is going on, let me make a few commnets based on what you have shown us. First using CODE tags instead of ICODE tags and reformatting your awk script, it looks like this:
Code:
BEGIN { good_count = 0
        bad_count = 0
        line_count = 0
}
NR == 1 {
        rec_count = substr($0, 15, 8)
        date_str = substr($0, 32, 10)

        print "Date in Header Record : " date_str
}
NR > 1 {line_count++
        if (length($0) == 710) {
                print $0 > GoodFile
                good_count++
        } else {
                print $0 > BadFile
                bad_count++
        }
}
END {   if(line_count == rec_count && bad_count==0) {
                print "********************************************"
                print "Count in header : " rec_count
                print "Count in file received : " line_count
                print " "
                print "Number of Good Records : " good_count
                print "Number of Bad Records : " bad_count
        } else {
                print "********************************************"
                print " PROBLEM WITH FILE SENT FROM SOURCE "
                print " "
                print "Count in header : " rec_count
                print "Count in file received : " line_count
                print " "
                print "Number of Good Records : " good_count
                print "Number of Bad Records : " bad_count
                exit 99
        }
}

You keep asking how to find out how many characters are in the first line of your input file, but (as shown by the lines marked in red) this awk script doesn't care what the length is for the first line. It is checking to see that the length of every line AFTER the first line contains 710 characters not counting the terminating <newline> character. It assumes that the first line in the file contains at least 43 characters, but does not verify that that is true.

If you don't know how this awk script is being run, how do you know it isn't working correctly???

The file you uploaded (First row of src file.txt) happens to have 712 characters in the 1st line (counting the trailing <carriage-return> and <newline> characters), but the second line is incomplete. Since the file you uploaded contained 1422 bytes (and only contained one <newline> character as the 712th character in the file), we have no idea how long the 2nd line in the file is. Since awk is only specified to work on text files, the behavior of this script is unspecified when given your uploaded file as input. On OS X, it yields:
Code:
Date in Header Record : 04/22/2013
********************************************
 PROBLEM WITH FILE SENT FROM SOURCE 
 
Count in header : 400133  
Count in file received : 1
 
Number of Good Records : 1
Number of Bad Records : 0

but if you add a <carriage-return> and a <newline> to the end of your uploaded file (matching what is at the end of the 1st line in the file), it yields:
Code:
Date in Header Record : 04/22/2013
********************************************
 PROBLEM WITH FILE SENT FROM SOURCE 
 
Count in header : 400133  
Count in file received : 1
 
Number of Good Records : 0
Number of Bad Records : 1

As I said all of this is based on lots of conjecture, but if you won't show us how the script is used, how the input file is massaged before feeding it to the script, and any other pertinent details that have been hidden by not showing us how this awk script is invoked; we can't make any concrete suggestions on what, if anything, might be wrong. Maybe the <carriage-return> characters in you input file are discarded before the input is fed into this awk script??? Maybe the only <carriage-return> in the input file is in the first line (where it will be ignored).???
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Convert variable length record to fixed length

Hi Team, I have an issue to split the file which is having special chracter(German Char) using awk command. I have a different length records in a file. I am separating the files based on the length using awk command. The command is working fine if the record is not having any... (7 Replies)
Discussion started by: Anthuvan
7 Replies

2. Shell Programming and Scripting

Record length

Hi, The record length may be differ in afile. I want to display the records if the record length is not equal to 50 using sed/awk command. Thanks in Advance (6 Replies)
Discussion started by: NareshN
6 Replies

3. Shell Programming and Scripting

Viewing a record of particular length

I have records with different lengths say 386, 387 and 388. Do i have any command to view all the records of the length 386 ? Please do advise. Thanks (2 Replies)
Discussion started by: bobby1015
2 Replies

4. Shell Programming and Scripting

Make variable length record a fixed length

Very, very new to unix scripting and have a unique situation. I have a file of records that contain 3 records types: (H)eader Records (D)etail Records (T)railer Records The Detail records are 82 bytes in length which is perfect. The Header and Trailer records sometimes are 82 bytes in... (3 Replies)
Discussion started by: jclanc8
3 Replies

5. Shell Programming and Scripting

Check length of record

Hi, I have a problem, please help me, I have a flat file like this: P00000000088888888999999999 0000999903 000000000000000000 P00000000077777777000000000 0000999903 000000000000000000 P00000000044444444333333333 0000999903 00000000000000000079875 P00000000066666666111111111 0000999903 ... (5 Replies)
Discussion started by: DebianJ
5 Replies

6. UNIX for Dummies Questions & Answers

What the command to find out the record length of a fixed length file?

I want to find out the record length of a fixed length file? I forgot the command. Any body know? (9 Replies)
Discussion started by: tranq01
9 Replies

7. Shell Programming and Scripting

AWK record length fix

Hi Friends, Need some help in AWK. Working on AIX 5 Have been trying the following functionality to make the record length fixed: if( length(record) < 300 ) { printf("%-300s\n", record); } In my opinion it will apply some fillers in the end. Its is not making any... (4 Replies)
Discussion started by: kanu_pathak
4 Replies

8. UNIX for Dummies Questions & Answers

Creating new file with new record length

Hi there, using the command below > cat file1 | tr "\001" " " > tmp2 how can I alter the record length of tmp2 to fix length of 350 bytes ? Thanks a lot! (7 Replies)
Discussion started by: mrjunsy
7 Replies

9. Shell Programming and Scripting

fixed record length

hello! I have a file with fixed record length... format: 123445asdfg 4343777 sfgg I wanna convert it to 123445,asdfg ,4343,777 ,sfgg is there any way to do it? sed/grep/awk?? at the moment I use sed -e 's_ \(\)_,\1_g' but it works only if there are spaces between... (16 Replies)
Discussion started by: george_
16 Replies
Login or Register to Ask a Question