Sponsored Content
Top Forums UNIX for Beginners Questions & Answers awk division without rounding Post 303046061 by RudiC on Wednesday 22nd of April 2020 03:46:27 PM
Old 04-22-2020
Decimal numbers represented by a binary system are ALWAYS rounded / subject to rounding. For example, your first number, given as 0.123456789012, is internally approximated / stored as 0.123456789011999995553381381796, which itself is a rounded value already. You may end up less than or greater than your target value, or, in rare cases, exactly on target (0.5, or 0.25, for example).

So, your request is a bit diffucult to fulfill. You can try awk's OFMT variable set to a fixed output field length, but, surprise, additional undesired decimal places occur:
Code:
awk '{val = $1 / 100; print val}' OFMT="%.20f" file
0.00123456789012000006
0.00123456789012344999
0.00123456789012345671


If that is unsatisfying, you can resort to formatted print and a tricky taylored field length computation:

Code:
awk '{val = $1 / 100; printf "%.*f\n", length($1), val}' file
0.00123456789012
0.00123456789012345
0.0012345678901234567

This works as there are two additional places needed for the division by 100, but length ($1) already counts two characters: the leading 0 and dot. If you have more significant leading digits, additional measures / steps need to be taken.
This User Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Annoying rounding issue in awk

Hello I am getting this very annoying issue in awk: awk '{a=12825;b=a*1.25; print b}' test 16031.2 Thing is the multiplication result is wrong... Result should be 16031.25. I think the issue only happens on bigger numbers. What can I do to get passed this? Thanks by advance (3 Replies)
Discussion started by: Indalecio
3 Replies

2. Shell Programming and Scripting

Rounding issue with awk

Hi Friends, I am trying to round following number. 0.07435000 echo "0.07435000"|awk '{printf "%s\n",$1*100}'|awk '{printf "%.2f\n",$1}' It returns: 7.435 It should return: 7.44 Any suggestion please? Thanks, Prashant (2 Replies)
Discussion started by: ppat7046
2 Replies

3. Shell Programming and Scripting

awk Division and modulus

I need to read the file divide 3 column with 2nd and run a modulus of 10 and check whether the remainder is zero or not if not print the entire line. cat filename | awk '{ if ($3 / $2 % 10 != 0) print $0}' Whats wrong with it ? (4 Replies)
Discussion started by: dinjo_jo
4 Replies

4. Shell Programming and Scripting

AWK rounding up numbers

Hi, I have managed to round up numbers by using the following command: echo "5.54" | awk '{printf "%.0f\n", $1}' result 6 How can I round up all the numbers in a column in a file and print the lines with the new calculated totals? Thanks, (3 Replies)
Discussion started by: keenboy100
3 Replies

5. UNIX for Advanced & Expert Users

awk: division by zero

I received error "awk: division by zero" while executing the following statement. SunOS 5.10 Generic_142900-15 sun4us sparc FJSV,GPUZC-M echo 8 | awk 'END {printf ("%d\n",NR/$1 + 0.5);}' file1.lst awk: division by zero Can someone provide solution? Thanks Please use code... (11 Replies)
Discussion started by: kumar77
11 Replies

6. Shell Programming and Scripting

awk, floating point and rounding

I had a person bring an interesting problem to me that appears to involve some sort of rounding inside awk. I've verified this with awk and nawk on Solaris as well as with gawk 3.1.5 on a Linux box. The original code fragment he brought me was thus: for (index=0; index < 1; index=index+.1) ... (4 Replies)
Discussion started by: mmyer2
4 Replies

7. Shell Programming and Scripting

awk & division

vmstat|awk '{print $3}'|tail -1 returns 6250511, but what I need is 24416, which is 6250511 divided by 256. Please advise. Thank you so much (2 Replies)
Discussion started by: Daniel Gate
2 Replies

8. Shell Programming and Scripting

awk calculation automatically rounding off the output

I have some calculation in my script which is similar to the below example . I find that sometimes when using large decimal digits, the output gets automatically rounded off and it is affecting the program. I am not able to understand what is happening here.. awk '{ a=6.32498922 a1=6.324... (5 Replies)
Discussion started by: wanderingmind16
5 Replies

9. Shell Programming and Scripting

awk - Division with condition

Hi Friends, I have an input file like this cat input chr1 100 200 1 2 chr1 120 130 na 1 chr1 140 160 1 na chr1 170 180 na na chr1 190 220 0 0 chr1 220 230 nd 1 chr2 330 400 1 nd chr2 410 450 nd nd chr3 500 700 1 1 I want to calculate the division of 4th and 5th columns. But, if... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

10. Shell Programming and Scripting

[awk] rounding a float number?

Heyas Trying to calculate the total size of a file by reading its bitrate. Code snippet: fs_expected() { # # Returns the expected filesize in bytes # pr_str() { ff=$(cat $TMP.info) d="${ff#*bitrate: }" echo "${d%%,*}" | $AWK '{print $1}' | head -n 1 } t_BYTERATE=$((... (9 Replies)
Discussion started by: sea
9 Replies
Complex(3o)							   OCaml library						       Complex(3o)

NAME
Complex - Complex numbers. Module Module Complex Documentation Module Complex : sig end Complex numbers. This module provides arithmetic operations on complex numbers. Complex numbers are represented by their real and imaginary parts (carte- sian representation). Each part is represented by a double-precision floating-point number (type float ). type t = { re : float ; im : float ; } The type of complex numbers. re is the real part and im the imaginary part. val zero : t The complex number 0 . val one : t The complex number 1 . val i : t The complex number i . val neg : t -> t Unary negation. val conj : t -> t Conjugate: given the complex x + i.y , returns x - i.y . val add : t -> t -> t Addition val sub : t -> t -> t Subtraction val mul : t -> t -> t Multiplication val inv : t -> t Multiplicative inverse ( 1/z ). val div : t -> t -> t Division val sqrt : t -> t Square root. The result x + i.y is such that x > 0 or x = 0 and y >= 0 . This function has a discontinuity along the negative real axis. val norm2 : t -> float Norm squared: given x + i.y , returns x^2 + y^2 . val norm : t -> float Norm: given x + i.y , returns sqrt(x^2 + y^2) . val arg : t -> float Argument. The argument of a complex number is the angle in the complex plane between the positive real axis and a line passing through zero and the number. This angle ranges from -pi to pi . This function has a discontinuity along the negative real axis. val polar : float -> float -> t polar norm arg returns the complex having norm norm and argument arg . val exp : t -> t Exponentiation. exp z returns e to the z power. val log : t -> t Natural logarithm (in base e ). val pow : t -> t -> t Power function. pow z1 z2 returns z1 to the z2 power. OCamldoc 2012-06-26 Complex(3o)
All times are GMT -4. The time now is 04:23 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy