Sample Data:
The script should validate that only 1 header and trailer exists. If more, raise exception.
The script should verify that total detail lines equal to the trailder record (Record_count)
I really appreciate if someone can provide me the script
Thanks
Moderator's Comments:
Please view this code tag video for how to use code tags when posting code and data.
Why don't you break the problem down into several parts?
1) Check if the header exist. Use grep & head
2) Check if the trailer record exist. Use grep & tail
3) Check if there is multiple records of header & trailers. Use grep & wc
Equally important requirement is to verify record count matches in detail line and the record count sum in the trailer. How can this be done?
The awk programme that I posted and a small bug -- didn't properly check for multiple trailers. This does better:
It checks both for the existence of exactly one header and one trailer, AND checks that the record count in the trailer matches the records observed. If either fails the exit code is non-zero to indicate that there is an error. If you need more precision, knowing exactly why there is an error, a longer programme can be used:
I am not familiar with what an 'ETL program' is, so I don't know if you can invoke awk or not.
If ETL is anything like and of the standard *NIX shells, you probably can do something like this:
@chedlee88-1 -- for small files, reading each three times might not have a noticeable impact, but if the input being verified is large, reading each three times might be so inefficient as not to be practical. It makes more sense to read the file once.
This programme will do the same thing, non-zero exit code, and write the failure reason onto standard err.
Hi All;
I am struggling to write a script that validates file header.
Header file would be like below with TAB separated
TRX # TYPE REF # Source Piece Code Destination Piece Code
every time I need to check the txt file if the header was same as above fields if validation success... (6 Replies)
I have a EBCDIC multi layout file which has a header record which is 21 bytes, The Detail records are 2427 bytes long and the trailer record is 9 bytes long.
Is there a command to remove the header as well as trailer record and read only the detail records while at the same time not altering... (1 Reply)
I need to split the file based on pattern from position 34-37 while retaining the header and trailer records in each individual split file
Also is it possible to output the TOM and PAT records in the same output file ?
I need the output file names same as xyz_pattern_Datetimestamp.txt
... (23 Replies)
Hi All,
I am trying write a simple command using AWK and SED to this but without any success.
Here is what I am using:
head -1 test1.txt>test2.txt|sed '1d;$d' test1.txt|awk '{print substr($0,0,(length($0)-2))}' >>test2.txt|tail -1 test1.txt>>test2.txt
Input:
Header
1234567
abcdefgh... (2 Replies)
Hi,
I want to add the below Header to all the files in sequence File1,File2,File3...etc
"ABC,<number of chracter in the file>"
e,g - If File1 is as below
pqrstuvdt
abcdefgh
then I want to add the above header into it ,So that File1 becomes as below
ABC,17
pqrstuvdt
abcdefgh
... (9 Replies)
How can we a shell script and pass date parameters .I have 3 files comming from Datastage with |" delimited
I need append 3 files as above: File1:
P0000|"47416954|"AU|"000|"INS|"0000|"|"20060601|"99991231|"|"|"|"|"01
File 2:... (2 Replies)
Hi all,
I have a requirement to create a Header &Trailer for a file which is having 20 millions of records.
If I use the following method, i think it will take more time.
cat "Header"> file1.txt
cat Data_File>>file1.txt
cat "Trailer">>file1.txt
since second CAT command has to read all... (4 Replies)
Hi All,
I am karthik. I am new to this forum. I have one requirement.
I have a file with header and footer.
Header may be like
HDR0001
or
FILE20090110
(Assume it is unknown so far, but i am sure there is a header in the file)
likewise file has the trailer too.
I just... (7 Replies)
All,
I am new to unix and i have the following requirement.
I have file(s) landing into input directory with timestamp, first i want to copy all these files into seperate directory then i want to rename these files without timestamp and also remove header,trailer from that file..
Could... (35 Replies)