Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk, floating point and rounding
Top Forums Shell Programming and Scripting awk, floating point and rounding Post 302488905 by mmyer2 on Tuesday 18th of January 2011 07:01:50 PM
Old 01-18-2011
awk, floating point and rounding
I had a person bring an interesting problem to me that appears to involve some sort of rounding inside awk. I've verified this with awk and nawk on Solaris as well as with gawk 3.1.5 on a Linux box.

The original code fragment he brought me was thus:

Code:
for (index=0; index < 1; index=index+.1)
                printf "Index=%f\n",index

What was odd was this would print an index of 1 even though the script should have found "index < 1" false when index==1.

I wrote a little follow on script to highlight this further

Code:
BEGIN   {
        for (i=5; i < 7; i+=1) {
                printf "Pass %s...\n",i
                for (j=0.0; j+0.0 < (.1*i); j=j+0.1) {
                        printf "j=%f\n",j;
                }
        }
exit
}'

I would expect the output of this to print 5 lines, then 6 lines but instead we see 5 lines then 7 lines:

Code:
Pass 5...
j=0.000000
j=0.100000
j=0.200000
j=0.300000
j=0.400000
Pass 6...
j=0.000000
j=0.100000
j=0.200000
j=0.300000
j=0.400000
j=0.500000
j=0.600000

The fact that the change occurs around .5 makes me think we're seeing some sort of integer rounding in action but I don't know how to force a variable to always be interpreted as a float to avoid this -- as best as I can tell awk doesn't support any for of typecasting. In the example you'll see that I always use a floating point value when doing math on the variables but that doesn't seem to affect things one way or another.

Any thoughts appreciated!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Floating Point Division

Does anyone have a simple way of doing floating point ("fp") division? For example, if I divide 3 by 5, I can get 0.6. The built-in calc (`bc`) will perform fp multiplication, but not division, at least not straight-up (i.e., starting bc and just typing in 3/5). I am trying to do this using... (1 Reply)
Discussion started by: gsatch
1 Replies

2. Shell Programming and Scripting

problem with floating point numbers in awk

hi all, i have the following problem using awk in a script i want to read the values from a column with real numbers and calculate the mean.the problem is that when i use a statement such as this num = $4 i cant find a way to convert the variable from string to floating point to perform... (7 Replies)
Discussion started by: kanagias
7 Replies

3. Shell Programming and Scripting

floating point addition

hi, :) I have a file like this 10.456 123.567 456.876 234.987 ........ ....... What i want to do is ia have to add all those numbers and put the result in some other file. Any help pls. cheers RRK (8 Replies)
Discussion started by: ravi raj kumar
8 Replies

4. Shell Programming and Scripting

Replace floating-point by integer in awk

Hi, I am trying to write a script to extract multiple sets of data from a chemistry output file. The problem section is in the following format... Geometry "geometry" -> "geometry" 1 Pd 46.0000 -0.19290971 0.00535260 0.02297606 2 P ... (7 Replies)
Discussion started by: smadonald1
7 Replies

5. Shell Programming and Scripting

Rounding off the value of Floating point value

Hello, i have some variables say: x=1.4 y=3.7 I wish to round off these values to : x = 2 (after rounding off) y = 4 (after rounding off) I am stuck. Please help. (7 Replies)
Discussion started by: damansingh
7 Replies

6. Linux

Floating Point Exception

Hi, I am compiling "HelloWorld" C progam on 32-bit CentOS and i want to execute it on 64-bit CentOS architecture. For that i copied the a.out file from 32-bit to 64-bit machine, but while executing a.out file on 64bit machine I am getting "Floating point exception error". But we can run... (3 Replies)
Discussion started by: Mandar123
3 Replies

7. Shell Programming and Scripting

how to compare 2 floating point no.

Hi, Could any one tell me how to compare to floating point no. using test command. As -eq option works on only intergers. i=5.4 if then echo "equal" else echo "not equal" fi here output will be equal even though no. are unequal. Thanks, ravi (1 Reply)
Discussion started by: useless79
1 Replies

8. Shell Programming and Scripting

Arithmetic in floating point

is it not possible to simply di aritmetic without using bc or awk i have tried folllowing operatrions but they support only integer types plz suggest me code for floating using values stored in the variables.the ans i get is integer and if i input floating values i get error numeric constant... (6 Replies)
Discussion started by: sumit the cool
6 Replies

9. Programming

Floating Point

Anyone help me i cant found the error of floating point if needed, i added the code complete #include <stdio.h> #include <string.h> #include <stdlib.h> #include <ctype.h> typedef struct { int hh; int mm; int ss; char nom; int punt; }cancion; typedef struct... (9 Replies)
Discussion started by: Slasho
9 Replies

10. Shell Programming and Scripting

Rounding off to the nearest floating number

I have a number, which I want to convert into the nearest floating number upto two places after the decimal point. E.g. 1.2346 will become 1.23 but 1.2356 will become 1.24 . Similarly 0.009 will be 0.01 and 0.001 will be 0.00 or 0.0 (not 0, wnat to keep the decimal... (1 Reply)
Discussion started by: hbar
1 Replies

LEARN ABOUT OPENSOLARIS

igawk

IGAWK(1)							 Utility Commands							  IGAWK(1)

NAME

       igawk - gawk with include files

SYNOPSIS

       igawk [ all gawk options ] -f program-file [ -- ] file ...
       igawk [ all gawk options ] [ -- ] program-text file ...

DESCRIPTION

       Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1).

       AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like

	      @include getopt.awk

       in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path.

OPTIONS

       See gawk(1) for a full description of the AWK language and the options that gawk supports.

EXAMPLES

       cat << EOF > test.awk
       @include getopt.awk

       BEGIN {
	    while (getopt(ARGC, ARGV, "am:q") != -1)
		 ...
       }
       EOF

       igawk -f test.awk

SEE ALSO

       gawk(1)

       Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995.

AUTHOR

       Arnold Robbins (arnold@skeeve.com).

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +--------------------+-----------------+
       |  ATTRIBUTE TYPE    | ATTRIBUTE VALUE |
       +--------------------+-----------------+
       |Availability	    | SUNWgawk	      |
       +--------------------+-----------------+
       |Interface Stability | Volatile	      |
       +--------------------+-----------------+
NOTES

       Source for gawk is available on http://opensolaris.org.

Free Software Foundation					    Nov 3 1999								  IGAWK(1)
All times are GMT -4. The time now is 06:09 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy