Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk, floating point and rounding

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk, floating point and rounding
# 1  
Old 01-18-2011
awk, floating point and rounding
I had a person bring an interesting problem to me that appears to involve some sort of rounding inside awk. I've verified this with awk and nawk on Solaris as well as with gawk 3.1.5 on a Linux box.

The original code fragment he brought me was thus:

Code:
for (index=0; index < 1; index=index+.1)
                printf "Index=%f\n",index

What was odd was this would print an index of 1 even though the script should have found "index < 1" false when index==1.

I wrote a little follow on script to highlight this further

Code:
BEGIN   {
        for (i=5; i < 7; i+=1) {
                printf "Pass %s...\n",i
                for (j=0.0; j+0.0 < (.1*i); j=j+0.1) {
                        printf "j=%f\n",j;
                }
        }
exit
}'

I would expect the output of this to print 5 lines, then 6 lines but instead we see 5 lines then 7 lines:

Code:
Pass 5...
j=0.000000
j=0.100000
j=0.200000
j=0.300000
j=0.400000
Pass 6...
j=0.000000
j=0.100000
j=0.200000
j=0.300000
j=0.400000
j=0.500000
j=0.600000

The fact that the change occurs around .5 makes me think we're seeing some sort of integer rounding in action but I don't know how to force a variable to always be interpreted as a float to avoid this -- as best as I can tell awk doesn't support any for of typecasting. In the example you'll see that I always use a floating point value when doing math on the variables but that doesn't seem to affect things one way or another.

Any thoughts appreciated!
# 2  
Old 01-18-2011
Hi

This works for me:


Code:
BEGIN   {
        for (i=5; i <=7; i+=1) {
                printf "Pass %s...\n",i
                for (j=0.0; (float)j < (.1*i); j=j+0.1) {
                        printf "j=%f\n",j;
                }
        }
exit
}


Guru.
# 3  
Old 01-18-2011
Indeed it does!

I looked for information on typecasting in the awk/nawk/gawk man pages but found none but apparently it works and it supported even in the ancient 'awk' that Solaris has by default.

Thanks for the pointer!
# 4  
Old 01-18-2011
There is nothing special about "(float)j". Awk interprets it as a concatenation of two strings. First the uninitialized variable "float" is treated as an empty string, then the variable j is converted to string using the default "%.6g" format.
Code:
awk 'BEGIN{for(x=0.1;x<1;x+=.1)printf("%.20f\n",x)}'
0.10000000000000000555
0.20000000000000001110
0.30000000000000004441
0.40000000000000002220
0.50000000000000000000
0.59999999999999997780
0.69999999999999995559
0.79999999999999993339
0.89999999999999991118
0.99999999999999988898

Here you can see that the last number is very close to 1. When it is converted to string using the default "%.6g", it will be 1 exactly.
# 5  
Old 01-19-2011
The "error" is due to the default rounding.
Code:
BEGIN   {
        for (i=5; i < 7; i+=1) {
                printf "Pass %s...\n",i
                for (j = 0.0; j < (0.1*i); j = j + 0.1) {
                        printf "j=%.20f %.20f\n", j, .1*i;
                }
        }
exit
}

Code:
Pass 5...
j=0.00000000000000000000 0.50000000000000000000
j=0.10000000000000000555 0.50000000000000000000
j=0.20000000000000001110 0.50000000000000000000
j=0.30000000000000004441 0.50000000000000000000
j=0.40000000000000002220 0.50000000000000000000
Pass 6...
j=0.00000000000000000000 0.60000000000000008882
j=0.10000000000000000555 0.60000000000000008882
j=0.20000000000000001110 0.60000000000000008882
j=0.30000000000000004441 0.60000000000000008882
j=0.40000000000000002220 0.60000000000000008882
j=0.50000000000000000000 0.60000000000000008882
j=0.59999999999999997780 0.60000000000000008882

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Rounding off to the nearest floating number

I have a number, which I want to convert into the nearest floating number upto two places after the decimal point. E.g. 1.2346 will become 1.23 but 1.2356 will become 1.24 . Similarly 0.009 will be 0.01 and 0.001 will be 0.00 or 0.0 (not 0, wnat to keep the decimal... (1 Reply)
Discussion started by: hbar
1 Replies

2. Programming

Floating Point

Anyone help me i cant found the error of floating point if needed, i added the code complete #include <stdio.h> #include <string.h> #include <stdlib.h> #include <ctype.h> typedef struct { int hh; int mm; int ss; char nom; int punt; }cancion; typedef struct... (9 Replies)
Discussion started by: Slasho
9 Replies

3. Shell Programming and Scripting

Arithmetic in floating point

is it not possible to simply di aritmetic without using bc or awk i have tried folllowing operatrions but they support only integer types plz suggest me code for floating using values stored in the variables.the ans i get is integer and if i input floating values i get error numeric constant... (6 Replies)
Discussion started by: sumit the cool
6 Replies

4. Shell Programming and Scripting

how to compare 2 floating point no.

Hi, Could any one tell me how to compare to floating point no. using test command. As -eq option works on only intergers. i=5.4 if then echo "equal" else echo "not equal" fi here output will be equal even though no. are unequal. Thanks, ravi (1 Reply)
Discussion started by: useless79
1 Replies

5. Linux

Floating Point Exception

Hi, I am compiling "HelloWorld" C progam on 32-bit CentOS and i want to execute it on 64-bit CentOS architecture. For that i copied the a.out file from 32-bit to 64-bit machine, but while executing a.out file on 64bit machine I am getting "Floating point exception error". But we can run... (3 Replies)
Discussion started by: Mandar123
3 Replies

6. Shell Programming and Scripting

Rounding off the value of Floating point value

Hello, i have some variables say: x=1.4 y=3.7 I wish to round off these values to : x = 2 (after rounding off) y = 4 (after rounding off) I am stuck. Please help. (7 Replies)
Discussion started by: damansingh
7 Replies

7. Shell Programming and Scripting

Replace floating-point by integer in awk

Hi, I am trying to write a script to extract multiple sets of data from a chemistry output file. The problem section is in the following format... Geometry "geometry" -> "geometry" 1 Pd 46.0000 -0.19290971 0.00535260 0.02297606 2 P ... (7 Replies)
Discussion started by: smadonald1
7 Replies

8. Shell Programming and Scripting

floating point addition

hi, :) I have a file like this 10.456 123.567 456.876 234.987 ........ ....... What i want to do is ia have to add all those numbers and put the result in some other file. Any help pls. cheers RRK (8 Replies)
Discussion started by: ravi raj kumar
8 Replies

9. Shell Programming and Scripting

problem with floating point numbers in awk

hi all, i have the following problem using awk in a script i want to read the values from a column with real numbers and calculate the mean.the problem is that when i use a statement such as this num = $4 i cant find a way to convert the variable from string to floating point to perform... (7 Replies)
Discussion started by: kanagias
7 Replies

10. Shell Programming and Scripting

Floating Point Division

Does anyone have a simple way of doing floating point ("fp") division? For example, if I divide 3 by 5, I can get 0.6. The built-in calc (`bc`) will perform fp multiplication, but not division, at least not straight-up (i.e., starting bc and just typing in 3/5). I am trying to do this using... (1 Reply)
Discussion started by: gsatch
1 Replies
Login or Register to Ask a Question