Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk, floating point and rounding
Top Forums Shell Programming and Scripting awk, floating point and rounding Post 302488905 by mmyer2 on Tuesday 18th of January 2011 07:01:50 PM
Old 01-18-2011
awk, floating point and rounding
I had a person bring an interesting problem to me that appears to involve some sort of rounding inside awk. I've verified this with awk and nawk on Solaris as well as with gawk 3.1.5 on a Linux box.

The original code fragment he brought me was thus:

Code:
for (index=0; index < 1; index=index+.1)
                printf "Index=%f\n",index

What was odd was this would print an index of 1 even though the script should have found "index < 1" false when index==1.

I wrote a little follow on script to highlight this further

Code:
BEGIN   {
        for (i=5; i < 7; i+=1) {
                printf "Pass %s...\n",i
                for (j=0.0; j+0.0 < (.1*i); j=j+0.1) {
                        printf "j=%f\n",j;
                }
        }
exit
}'

I would expect the output of this to print 5 lines, then 6 lines but instead we see 5 lines then 7 lines:

Code:
Pass 5...
j=0.000000
j=0.100000
j=0.200000
j=0.300000
j=0.400000
Pass 6...
j=0.000000
j=0.100000
j=0.200000
j=0.300000
j=0.400000
j=0.500000
j=0.600000

The fact that the change occurs around .5 makes me think we're seeing some sort of integer rounding in action but I don't know how to force a variable to always be interpreted as a float to avoid this -- as best as I can tell awk doesn't support any for of typecasting. In the example you'll see that I always use a floating point value when doing math on the variables but that doesn't seem to affect things one way or another.

Any thoughts appreciated!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Floating Point Division

Does anyone have a simple way of doing floating point ("fp") division? For example, if I divide 3 by 5, I can get 0.6. The built-in calc (`bc`) will perform fp multiplication, but not division, at least not straight-up (i.e., starting bc and just typing in 3/5). I am trying to do this using... (1 Reply)
Discussion started by: gsatch
1 Replies

2. Shell Programming and Scripting

problem with floating point numbers in awk

hi all, i have the following problem using awk in a script i want to read the values from a column with real numbers and calculate the mean.the problem is that when i use a statement such as this num = $4 i cant find a way to convert the variable from string to floating point to perform... (7 Replies)
Discussion started by: kanagias
7 Replies

3. Shell Programming and Scripting

floating point addition

hi, :) I have a file like this 10.456 123.567 456.876 234.987 ........ ....... What i want to do is ia have to add all those numbers and put the result in some other file. Any help pls. cheers RRK (8 Replies)
Discussion started by: ravi raj kumar
8 Replies

4. Shell Programming and Scripting

Replace floating-point by integer in awk

Hi, I am trying to write a script to extract multiple sets of data from a chemistry output file. The problem section is in the following format... Geometry "geometry" -> "geometry" 1 Pd 46.0000 -0.19290971 0.00535260 0.02297606 2 P ... (7 Replies)
Discussion started by: smadonald1
7 Replies

5. Shell Programming and Scripting

Rounding off the value of Floating point value

Hello, i have some variables say: x=1.4 y=3.7 I wish to round off these values to : x = 2 (after rounding off) y = 4 (after rounding off) I am stuck. Please help. (7 Replies)
Discussion started by: damansingh
7 Replies

6. Linux

Floating Point Exception

Hi, I am compiling "HelloWorld" C progam on 32-bit CentOS and i want to execute it on 64-bit CentOS architecture. For that i copied the a.out file from 32-bit to 64-bit machine, but while executing a.out file on 64bit machine I am getting "Floating point exception error". But we can run... (3 Replies)
Discussion started by: Mandar123
3 Replies

7. Shell Programming and Scripting

how to compare 2 floating point no.

Hi, Could any one tell me how to compare to floating point no. using test command. As -eq option works on only intergers. i=5.4 if then echo "equal" else echo "not equal" fi here output will be equal even though no. are unequal. Thanks, ravi (1 Reply)
Discussion started by: useless79
1 Replies

8. Shell Programming and Scripting

Arithmetic in floating point

is it not possible to simply di aritmetic without using bc or awk i have tried folllowing operatrions but they support only integer types plz suggest me code for floating using values stored in the variables.the ans i get is integer and if i input floating values i get error numeric constant... (6 Replies)
Discussion started by: sumit the cool
6 Replies

9. Programming

Floating Point

Anyone help me i cant found the error of floating point if needed, i added the code complete #include <stdio.h> #include <string.h> #include <stdlib.h> #include <ctype.h> typedef struct { int hh; int mm; int ss; char nom; int punt; }cancion; typedef struct... (9 Replies)
Discussion started by: Slasho
9 Replies

10. Shell Programming and Scripting

Rounding off to the nearest floating number

I have a number, which I want to convert into the nearest floating number upto two places after the decimal point. E.g. 1.2346 will become 1.23 but 1.2356 will become 1.24 . Similarly 0.009 will be 0.01 and 0.001 will be 0.00 or 0.0 (not 0, wnat to keep the decimal... (1 Reply)
Discussion started by: hbar
1 Replies

LEARN ABOUT DEBIAN

pinot-index

PINOT-INDEX(1)							   User Commands						    PINOT-INDEX(1)

NAME

       pinot-index - Index documents from the command-line

SYNOPSIS

       pinot-index [OPTIONS] --db DATABASE URLS

DESCRIPTION

       pinot-index - Index documents from the command-line

OPTIONS

       -b, --backend
	      name of back-end to use (default xapian)

       -c, --check
	      check whether the given URL is in the index

       -d, --db
	      path to, or name of, index to use (mandatory)

       -h, --help
	      display this help and exit

       -i, --index
	      index the given URL

       -s, --showinfo
	      show information about the document

       -v, --version
	      output version information and exit

       Supported back-ends are : 'xapian'

EXAMPLES

       pinot-index --check --showinfo --backend xapian --db ~/.pinot/daemon ../Bozo.txt

       pinot-index --index --db PinotOnTheWeb http://code.google.com/p/pinot-search/

       Indexing documents to My Web Pages or My Documents with pinot-index is not recommended

REPORTING BUGS

       Report bugs to fabrice.colin@gmail.com

       This   is   free   software.    You   may   redistribute   copies   of	it   under   the   terms   of	the  GNU  General  Public  License
       <http://www.gnu.org/licenses/old-licenses/gpl-2.0.html>.
       There is NO WARRANTY, to the extent permitted by law.

pinot-index - pinot 1.0 					     June 2012							    PINOT-INDEX(1)
All times are GMT -4. The time now is 07:10 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy