Validating the record count


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Validating the record count
# 1  
Old 05-30-2012
Validating the record count

Hi ,

I am having a text file with several records., it has a header record and trailer record. The header record has the number of rows (records) found in the text file and time-stamp. The footer record has the total number of records ( along with the header and trailer.,

Suppose:
wc -l abc.txt -> Returns 150 (inclusive of header and trailer record)

#EOF -> Trailer record has count 148 . I want a script that validates this that exactly matches., since i have several text files in a folder and manually validating takes time. Please advice.
# 2  
Old 05-30-2012
Code:
awk '/^#EOF/ { if($NF != (NR-2)) exit 1 }' filename ||
        echo "Filename has wrong number of records"

This User Gave Thanks to Corona688 For This Post:
# 3  
Old 05-30-2012
Thanks Corona, can we include this inside some loop ., to test it for all files in the folder in one shot., and the output is directed to a log which has the details saying the particular file name has wrong number of records.
# 4  
Old 05-30-2012
Code:
$ cat floop.awk

# Special case for first line
NR==1 { F=FILENAME }
# Check counts and reset for every file
F != FILENAME {
        if(LFNR != (COUNT+2))   print F;
        LFNR=0; F=FILENAME
}
# Grab the count on the #EOF line
/^#EOF/ { COUNT=$NF }
# Update the count every line
{ LFNR=FNR }
# Check the very last file
END {   if(LFNR != (COUNT+2)) print F;  }

$ awk -f floop.awk data*

file2

$

# 5  
Old 05-30-2012
I had modified your first part.,
Code:
$ for f in $( ls #path of folder);do
  awk '/^#EOF/ { if($NF != (NR-2)) exit 1 }' f || echo " &f Filename has wrong number of records"
done

Can you please inform me how can i include else part here that check and logs that this files name has correct number of records., and also i need to redirect to a file (all the file validation results to one file)

Last edited by Franklin52; 05-31-2012 at 03:48 AM.. Reason: Please use code tags for data and code samples
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Need record count on every 30 minute

We have the below records where we need record count of every 30 minute like 00:01 to 00:30 so in that we will have 48 record count in 24 hrs , and also we need sum of record count from 00:01 to 23:30. Please find sample data as well. 00:01 21 00:02 23 00:03 34 00:04 34 00:05 30... (10 Replies)
Discussion started by: nadeemrafikhan
10 Replies

2. Shell Programming and Scripting

count of record in files

Hi all, I have written a scripts which count number of lines in all the files in a directory and write in a text file. I don't know how to format it while writing. OS suns solaris 10 my scripts is as below for i in /ersdg3/ERS/ERS_INPUT_LOGS/RIO/LOGS/RIO_02-Aug-2012/ *.LOG do echo... (11 Replies)
Discussion started by: guddu_12
11 Replies

3. UNIX for Dummies Questions & Answers

Reduce the count on the 98 record by 2,

recod,amount,noofaccount,count 98,+00000187865779787,00319,000000640/ 99,+00000187865779787,00001,000000642/ thsi is my input file my question is 1) Reduce the count on the 98 record by 2 (6 Replies)
Discussion started by: sgoud
6 Replies

4. Shell Programming and Scripting

Character count per record

I have a flat file. How can i retrive the character count per record in whole file. Can anybody assist me on this Cheers (9 Replies)
Discussion started by: subrat
9 Replies

5. Shell Programming and Scripting

Validate record count

Hi all How to verify the number of records in case of delimited file, If the file has records. And then if it is not equal to mentioned no. of records, email is triggered and file is moved to bad directory path. Example ----- input file = a.txt bad directory path : /usr/bin/bad (6 Replies)
Discussion started by: balaji23_d
6 Replies

6. UNIX for Advanced & Expert Users

character count per record

Hello can someone please advise. I need to send records in a file that are over 10,000 characters long to a seperate file. Any ideas? Thanks (2 Replies)
Discussion started by: Dolph
2 Replies

7. Shell Programming and Scripting

replaying a record count with another record count

i use unix command to take the record count for a file1 awk 'END{print NR}' filename i already have a file2 which conatin the count like ... .. rec_cnt=100 .. .. I want to replace the record in the file2 using the record i take from file1. suggest me some simple ways of doing it... (2 Replies)
Discussion started by: er_zeeshan05
2 Replies

8. Shell Programming and Scripting

record count

i have a file named file_names.dat where there are several files along with their path. exp: /data1/dir1/CTA_ACD_20071208.dat /data1/dir1/CTA_DFG_20071208.dat i want to write a script which will create a csv file with the name of the file and record count of that file the output file... (4 Replies)
Discussion started by: dr46014
4 Replies

9. Shell Programming and Scripting

Need help with Isql record count

What I am trying to do is check if the database query returned any records. If no records returned then output a message else output results to a file. Right now if I take out the if and else statements the code runs fine and sends the email. If no records returned the email sends the column... (4 Replies)
Discussion started by: johnu122
4 Replies

10. UNIX for Dummies Questions & Answers

How to count the record count in an EBCDIC file.

How do I get the record count in an EBCDIC file on a Linux Box. :confused: (1 Reply)
Discussion started by: oracle8
1 Replies
Login or Register to Ask a Question