Why sum of recs in awk don't match total rec count?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Why sum of recs in awk don't match total rec count?
# 1  
Old 02-26-2013
Why sum of recs in awk don't match total rec count?

I'm using awk to determine if a field starting in position 604 for length of 10 is not equal to ALL spaces. It's searching several files which are in the current directory.

The below awk indicates that there are 84 records on all files where this field IS NOT equal to ALL spaces ( there are 10 spaces between the quotes).

Code:
 
awk 'substr($0,604,10)!="          " {print substr($0,604,10)}' LPM_BENE_BATCH_*.txt | wc -l
84

I then want to run the negate of the above to validate against the total record count of all files.
The below awk indicates that there are 444 records on all files where this field equals all spaces.

Code:
 
awk 'substr($0,604,10)=="          " {print substr($0,604,10)}' LPM_BENE_BATCH_*.txt | wc -l
444

But when I count all records on all the same files the above 2 awk commands are searching I get 1 less record then when I sum the above counts (84+444 = 528).

Code:
 
wc -l LPM_BENE_BATCH_*.txt 
527 total

What are some factors that could cause my total to not match?
# 2  
Old 02-26-2013
wc counts the number of newlines. If one of the files doesn't end in a newline, that would explain the discrepancy, since awk would add the missing newline when it prints that record.

Regards,
Alister
This User Gave Thanks to alister For This Post:
# 3  
Old 02-26-2013
Thanks for the explanation alister! I was able to find the offending line missing the newline char.

Last edited by mjf; 02-26-2013 at 09:45 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Awk: count unique elements in a field and sum their occurence across the entire file

Hi, Sure it's an easy one, but it drives me insane. input ("|" separated): 1|A,B,C,A 2|A,D,D 3|A,B,B I would like to count the occurence of each capital letters in $2 across the entire file, knowing that duplicates in each record count as 1. I am trying to get this output... (5 Replies)
Discussion started by: beca123456
5 Replies

2. Shell Programming and Scripting

awk to output match and mismatch with count using specific fields

In the below awk I am trying output to one file those lines that match between $2,$3,$4 of file1 and file2 with the count in (). I am also trying to output those lines that are missing between $2,$3,$4 of file1 and file2 with the count of in () each. Both input files are tab-delimited, but the... (7 Replies)
Discussion started by: cmccabe
7 Replies

3. Shell Programming and Scripting

awk pattern match and count unique in column

Hi all I have a need of searching some pattern in file by month and then count unique records D11 G11 R11 -------> Pattern available in file S11 Jan$1 to $5 column contains some records in which I want to find unique for this purpose I have written script like below awk '/Jan/ ||... (4 Replies)
Discussion started by: nex_asp
4 Replies

4. Shell Programming and Scripting

awk Help -- If match found return the count

Hi All, I need to get the count of records in the file, if the passing parameter matches with the list of records in the file. Below is my example source file: Test1.dat 20120913 20120913 20120912 20120912 20120912 20120912 20120912 20120913 20120913 20120912 In my script I am... (5 Replies)
Discussion started by: bbc17484
5 Replies

5. Shell Programming and Scripting

sum using awk with pattern match

I have a file which has data like this *** Query completed. One row found. *** Query completed. One row found. *** Query completed. One row found. *** Insert completed. 5 rows added. *** Query completed. No rows found. *** Query completed. One row found. *** Query completed. One... (2 Replies)
Discussion started by: sol_nov
2 Replies

6. Shell Programming and Scripting

awk and count sum ?

I have a input.txt file which have 3 fields separate by a comma place, os and timediff in seconds tampa,win7, 2575 tampa,win7, 157619 tampa,win7, 3352 dallas,vista,604799 greenbay,winxp, 14400 greenbay,win7 , 518400 san jose,winxp, 228121 san jose,winxp, 70853 san jose,winxp, 193514... (5 Replies)
Discussion started by: sabercats
5 Replies

7. Shell Programming and Scripting

Total Count using AWK

Hi Everybody, I have the following example file... 199|TST-GURGAON|GURGAON|1 199|TST-GURGAON|GURGAON|1 199|TST-GURGAON|GURGAON|1 199|TST-GURGAON|GURGAON|1 199|TST-GURGAON|GURGAON|1 199|TST-GURGAON|GURGAON|1 199|TST-GURGAON|GURGAON|1 199|TST-GURGAON|GURGAON|1 199|TST-GURGAON|GURGAON|1... (8 Replies)
Discussion started by: sraj142
8 Replies

8. Shell Programming and Scripting

awk count characters, sum, and divide by another column

Hi All, I am another biologist attempting to parse a large txt file containing several million lines like: tucosnp 56762 T Y 228 228 60 23 .CcCcc,,..c.c,cc,,.C... What I need to do is get the frequency of periods (.) plus commas (,) in column 9, and populate this number into another... (1 Reply)
Discussion started by: peromhc
1 Replies

9. UNIX for Dummies Questions & Answers

how to count number of rows and sum of column using awk

Hi All, I have the following input which i want to process using AWK. Rows,NC,amount 1,1202,0.192387 2,1201,0.111111 3,1201,0.123456 i want the following output count of rows = 3 ,sum of amount = 0.426954 Many thanks (2 Replies)
Discussion started by: pistachio
2 Replies
Login or Register to Ask a Question