Sponsored Content
Full Discussion: floating point problem
Top Forums Programming floating point problem Post 302109780 by Hitori on Thursday 8th of March 2007 03:18:27 AM
Old 03-08-2007
In memory floating point numbers (and other internal numbers excluding special cases like BCD) are stored in binary format (e.g. in Intel reverse order for integers on PC platforms), so 0.1, 0.01, ... cannot be represented exactly, only division by 2^x (not by 10 or any other number) can give you an exact number.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

problem with floating point numbers in awk

hi all, i have the following problem using awk in a script i want to read the values from a column with real numbers and calculate the mean.the problem is that when i use a statement such as this num = $4 i cant find a way to convert the variable from string to floating point to perform... (7 Replies)
Discussion started by: kanagias
7 Replies

2. Shell Programming and Scripting

floating point addition

hi, :) I have a file like this 10.456 123.567 456.876 234.987 ........ ....... What i want to do is ia have to add all those numbers and put the result in some other file. Any help pls. cheers RRK (8 Replies)
Discussion started by: ravi raj kumar
8 Replies

3. Linux

Floating point exception !!!

Hi, I have linux fedora 4 ver., 2.6 kernal. And qmail & mysql & samba servers are already configured on this server. When I try to install any package like squidguard ,dansguardian,webmin,rsnapshots with command rpm -ivh . It is giving error as “Floating point exception" Snap View is... (3 Replies)
Discussion started by: ssk01
3 Replies

4. Linux

Floating Point Exception

Hi, I am compiling "HelloWorld" C progam on 32-bit CentOS and i want to execute it on 64-bit CentOS architecture. For that i copied the a.out file from 32-bit to 64-bit machine, but while executing a.out file on 64bit machine I am getting "Floating point exception error". But we can run... (3 Replies)
Discussion started by: Mandar123
3 Replies

5. Programming

Floating point Emulator

what is floating point emulator(FPE)? where and why it is used? (1 Reply)
Discussion started by: pgmfourms
1 Replies

6. Shell Programming and Scripting

how to compare 2 floating point no.

Hi, Could any one tell me how to compare to floating point no. using test command. As -eq option works on only intergers. i=5.4 if then echo "equal" else echo "not equal" fi here output will be equal even though no. are unequal. Thanks, ravi (1 Reply)
Discussion started by: useless79
1 Replies

7. Shell Programming and Scripting

floating point numbers in if

# if > then > echo "1" > else > echo "2" > fi -bash: How can i compare floating point numbers inside statement? (15 Replies)
Discussion started by: proactiveaditya
15 Replies

8. Shell Programming and Scripting

floating point number problem

Hello folks I Hope everyone is fine. I am calculating number of bytes calculation from apache web log. awk '{ sum += $10 } END { print sum }' /var/httpd/log/mydomain.log 7.45557e+09 it show above number, what should i do it sow number like 7455, i mean if after decimal point above 5 it... (5 Replies)
Discussion started by: learnbash
5 Replies

9. Shell Programming and Scripting

problem with floating point number loops

Hey, I guess I am just to stupid and am not seeing the "wood for the trees", but I am always getting strange errors. I want to create a mesh with coordinates like: x y z 3.1 3.0 0.75 0 0 1 3.1 2.9 0.75 0 0 1 3.1 2.8 0.75 0 0 1 3.1 2.7 0.75 0 0 1 3.0 ... (10 Replies)
Discussion started by: ergy1983
10 Replies

10. Programming

Floating Point

Anyone help me i cant found the error of floating point if needed, i added the code complete #include <stdio.h> #include <string.h> #include <stdlib.h> #include <ctype.h> typedef struct { int hh; int mm; int ss; char nom; int punt; }cancion; typedef struct... (9 Replies)
Discussion started by: Slasho
9 Replies
Int32(3o)							   OCaml library							 Int32(3o)

NAME
Int32 - 32-bit integers. Module Module Int32 Documentation Module Int32 : sig end 32-bit integers. This module provides operations on the type int32 of signed 32-bit integers. Unlike the built-in int type, the type int32 is guaranteed to be exactly 32-bit wide on all platforms. All arithmetic operations over int32 are taken modulo 2^{32. Performance notice: values of type int32 occupy more memory space than values of type int , and arithmetic operations on int32 are gener- ally slower than those on int . Use int32 only when the application requires exact 32-bit arithmetic. val zero : int32 The 32-bit integer 0. val one : int32 The 32-bit integer 1. val minus_one : int32 The 32-bit integer -1. val neg : int32 -> int32 Unary negation. val add : int32 -> int32 -> int32 Addition. val sub : int32 -> int32 -> int32 Subtraction. val mul : int32 -> int32 -> int32 Multiplication. val div : int32 -> int32 -> int32 Integer division. Raise Division_by_zero if the second argument is zero. This division rounds the real quotient of its arguments towards zero, as specified for Pervasives.(/) . val rem : int32 -> int32 -> int32 Integer remainder. If y is not zero, the result of Int32.rem x y satisfies the following property: x = Int32.add (Int32.mul (Int32.div x y) y) (Int32.rem x y) . If y = 0 , Int32.rem x y raises Division_by_zero . val succ : int32 -> int32 Successor. Int32.succ x is Int32.add x Int32.one . val pred : int32 -> int32 Predecessor. Int32.pred x is Int32.sub x Int32.one . val abs : int32 -> int32 Return the absolute value of its argument. val max_int : int32 The greatest representable 32-bit integer, 2^{31 - 1. val min_int : int32 The smallest representable 32-bit integer, -2^{31. val logand : int32 -> int32 -> int32 Bitwise logical and. val logor : int32 -> int32 -> int32 Bitwise logical or. val logxor : int32 -> int32 -> int32 Bitwise logical exclusive or. val lognot : int32 -> int32 Bitwise logical negation val shift_left : int32 -> int -> int32 Int32.shift_left x y shifts x to the left by y bits. The result is unspecified if y < 0 or y >= 32 . val shift_right : int32 -> int -> int32 Int32.shift_right x y shifts x to the right by y bits. This is an arithmetic shift: the sign bit of x is replicated and inserted in the vacated bits. The result is unspecified if y < 0 or y >= 32 . val shift_right_logical : int32 -> int -> int32 Int32.shift_right_logical x y shifts x to the right by y bits. This is a logical shift: zeroes are inserted in the vacated bits regardless of the sign of x . The result is unspecified if y < 0 or y >= 32 . val of_int : int -> int32 Convert the given integer (type int ) to a 32-bit integer (type int32 ). val to_int : int32 -> int Convert the given 32-bit integer (type int32 ) to an integer (type int ). On 32-bit platforms, the 32-bit integer is taken modulo 2^{31, i.e. the high-order bit is lost during the conversion. On 64-bit platforms, the conversion is exact. val of_float : float -> int32 Convert the given floating-point number to a 32-bit integer, discarding the fractional part (truncate towards 0). The result of the con- version is undefined if, after truncation, the number is outside the range [ Int32.min_int , Int32.max_int ]. val to_float : int32 -> float Convert the given 32-bit integer to a floating-point number. val of_string : string -> int32 Convert the given string to a 32-bit integer. The string is read in decimal (by default) or in hexadecimal, octal or binary if the string begins with 0x , 0o or 0b respectively. Raise Failure int_of_string if the given string is not a valid representation of an integer, or if the integer represented exceeds the range of integers representable in type int32 . val to_string : int32 -> string Return the string representation of its argument, in signed decimal. val bits_of_float : float -> int32 Return the internal representation of the given float according to the IEEE 754 floating-point ``single format'' bit layout. Bit 31 of the result represents the sign of the float; bits 30 to 23 represent the (biased) exponent; bits 22 to 0 represent the mantissa. val float_of_bits : int32 -> float Return the floating-point number whose internal representation, according to the IEEE 754 floating-point ``single format'' bit layout, is the given int32 . type t = int32 An alias for the type of 32-bit integers. val compare : t -> t -> int The comparison function for 32-bit integers, with the same specification as Pervasives.compare . Along with the type t , this function compare allows the module Int32 to be passed as argument to the functors Set.Make and Map.Make . OCamldoc 2012-06-26 Int32(3o)
All times are GMT -4. The time now is 03:09 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy