Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk, floating point and rounding
Top Forums Shell Programming and Scripting awk, floating point and rounding Post 302488905 by mmyer2 on Tuesday 18th of January 2011 07:01:50 PM
Old 01-18-2011
awk, floating point and rounding
I had a person bring an interesting problem to me that appears to involve some sort of rounding inside awk. I've verified this with awk and nawk on Solaris as well as with gawk 3.1.5 on a Linux box.

The original code fragment he brought me was thus:

Code:
for (index=0; index < 1; index=index+.1)
                printf "Index=%f\n",index

What was odd was this would print an index of 1 even though the script should have found "index < 1" false when index==1.

I wrote a little follow on script to highlight this further

Code:
BEGIN   {
        for (i=5; i < 7; i+=1) {
                printf "Pass %s...\n",i
                for (j=0.0; j+0.0 < (.1*i); j=j+0.1) {
                        printf "j=%f\n",j;
                }
        }
exit
}'

I would expect the output of this to print 5 lines, then 6 lines but instead we see 5 lines then 7 lines:

Code:
Pass 5...
j=0.000000
j=0.100000
j=0.200000
j=0.300000
j=0.400000
Pass 6...
j=0.000000
j=0.100000
j=0.200000
j=0.300000
j=0.400000
j=0.500000
j=0.600000

The fact that the change occurs around .5 makes me think we're seeing some sort of integer rounding in action but I don't know how to force a variable to always be interpreted as a float to avoid this -- as best as I can tell awk doesn't support any for of typecasting. In the example you'll see that I always use a floating point value when doing math on the variables but that doesn't seem to affect things one way or another.

Any thoughts appreciated!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Floating Point Division

Does anyone have a simple way of doing floating point ("fp") division? For example, if I divide 3 by 5, I can get 0.6. The built-in calc (`bc`) will perform fp multiplication, but not division, at least not straight-up (i.e., starting bc and just typing in 3/5). I am trying to do this using... (1 Reply)
Discussion started by: gsatch
1 Replies

2. Shell Programming and Scripting

problem with floating point numbers in awk

hi all, i have the following problem using awk in a script i want to read the values from a column with real numbers and calculate the mean.the problem is that when i use a statement such as this num = $4 i cant find a way to convert the variable from string to floating point to perform... (7 Replies)
Discussion started by: kanagias
7 Replies

3. Shell Programming and Scripting

floating point addition

hi, :) I have a file like this 10.456 123.567 456.876 234.987 ........ ....... What i want to do is ia have to add all those numbers and put the result in some other file. Any help pls. cheers RRK (8 Replies)
Discussion started by: ravi raj kumar
8 Replies

4. Shell Programming and Scripting

Replace floating-point by integer in awk

Hi, I am trying to write a script to extract multiple sets of data from a chemistry output file. The problem section is in the following format... Geometry "geometry" -> "geometry" 1 Pd 46.0000 -0.19290971 0.00535260 0.02297606 2 P ... (7 Replies)
Discussion started by: smadonald1
7 Replies

5. Shell Programming and Scripting

Rounding off the value of Floating point value

Hello, i have some variables say: x=1.4 y=3.7 I wish to round off these values to : x = 2 (after rounding off) y = 4 (after rounding off) I am stuck. Please help. (7 Replies)
Discussion started by: damansingh
7 Replies

6. Linux

Floating Point Exception

Hi, I am compiling "HelloWorld" C progam on 32-bit CentOS and i want to execute it on 64-bit CentOS architecture. For that i copied the a.out file from 32-bit to 64-bit machine, but while executing a.out file on 64bit machine I am getting "Floating point exception error". But we can run... (3 Replies)
Discussion started by: Mandar123
3 Replies

7. Shell Programming and Scripting

how to compare 2 floating point no.

Hi, Could any one tell me how to compare to floating point no. using test command. As -eq option works on only intergers. i=5.4 if then echo "equal" else echo "not equal" fi here output will be equal even though no. are unequal. Thanks, ravi (1 Reply)
Discussion started by: useless79
1 Replies

8. Shell Programming and Scripting

Arithmetic in floating point

is it not possible to simply di aritmetic without using bc or awk i have tried folllowing operatrions but they support only integer types plz suggest me code for floating using values stored in the variables.the ans i get is integer and if i input floating values i get error numeric constant... (6 Replies)
Discussion started by: sumit the cool
6 Replies

9. Programming

Floating Point

Anyone help me i cant found the error of floating point if needed, i added the code complete #include <stdio.h> #include <string.h> #include <stdlib.h> #include <ctype.h> typedef struct { int hh; int mm; int ss; char nom; int punt; }cancion; typedef struct... (9 Replies)
Discussion started by: Slasho
9 Replies

10. Shell Programming and Scripting

Rounding off to the nearest floating number

I have a number, which I want to convert into the nearest floating number upto two places after the decimal point. E.g. 1.2346 will become 1.23 but 1.2356 will become 1.24 . Similarly 0.009 will be 0.01 and 0.001 will be 0.00 or 0.0 (not 0, wnat to keep the decimal... (1 Reply)
Discussion started by: hbar
1 Replies

LEARN ABOUT HPUX

ucblinks

ucblinks(1B)					     SunOS/BSD Compatibility Package Commands					      ucblinks(1B)

NAME

       ucblinks - adds /dev entries to give SunOS 4.x compatible names to SunOS 5.x devices

SYNOPSIS

       /usr/ucb/ucblinks [-e rulebase] [-r rootdir]

DESCRIPTION

       ucblinks  creates symbolic links under the /dev directory for devices whose SunOS 5.x names differ from their SunOS 4.x names. Where possi-
       ble, these symbolic links point to the device's SunOS 5.x name rather than to the actual /devices entry.

       ucblinks does not remove unneeded compatibility links; these must be removed by hand.

       ucblinks should be called each time the system is reconfiguration-booted, after any new SunOS 5.x links that are needed have been  created,
       since the reconfiguration may have resulted in more compatibility names being needed.

       In  releases prior to SunOS 5.4, ucblinks used a  nawk rule-base to construct the SunOS 4.x compatible names. ucblinks no longer uses  nawk
       for the default operation, although  nawk rule-bases can still be specifed with the -e option.  The  nawk rule-base equivalent to the SunOS
       5.4 default operation can be found in /usr/ucblib/ucblinks.awk.

OPTIONS

       -e rulebase     Specify rulebase as the file containing nawk(1) pattern-action statements.

       -r rootdir      Specify rootdir as the directory under which dev and devices will be found, rather than the standard root directory /.

FILES

       /usr/ucblib/ucblinks.awk        sample rule-base for compatibility links

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWscpu			   |
       +-----------------------------+-----------------------------+

SEE ALSO

       devlinks(1M), disks(1M), ports(1M), tapes(1M), attributes(5)

SunOS 5.10							    13 Apr 1994 						      ucblinks(1B)
All times are GMT -4. The time now is 03:31 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy