Sponsored Content
Top Forums UNIX for Beginners Questions & Answers awk division without rounding Post 303046061 by RudiC on Wednesday 22nd of April 2020 03:46:27 PM
Old 04-22-2020
Decimal numbers represented by a binary system are ALWAYS rounded / subject to rounding. For example, your first number, given as 0.123456789012, is internally approximated / stored as 0.123456789011999995553381381796, which itself is a rounded value already. You may end up less than or greater than your target value, or, in rare cases, exactly on target (0.5, or 0.25, for example).

So, your request is a bit diffucult to fulfill. You can try awk's OFMT variable set to a fixed output field length, but, surprise, additional undesired decimal places occur:
Code:
awk '{val = $1 / 100; print val}' OFMT="%.20f" file
0.00123456789012000006
0.00123456789012344999
0.00123456789012345671


If that is unsatisfying, you can resort to formatted print and a tricky taylored field length computation:

Code:
awk '{val = $1 / 100; printf "%.*f\n", length($1), val}' file
0.00123456789012
0.00123456789012345
0.0012345678901234567

This works as there are two additional places needed for the division by 100, but length ($1) already counts two characters: the leading 0 and dot. If you have more significant leading digits, additional measures / steps need to be taken.
This User Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Annoying rounding issue in awk

Hello I am getting this very annoying issue in awk: awk '{a=12825;b=a*1.25; print b}' test 16031.2 Thing is the multiplication result is wrong... Result should be 16031.25. I think the issue only happens on bigger numbers. What can I do to get passed this? Thanks by advance (3 Replies)
Discussion started by: Indalecio
3 Replies

2. Shell Programming and Scripting

Rounding issue with awk

Hi Friends, I am trying to round following number. 0.07435000 echo "0.07435000"|awk '{printf "%s\n",$1*100}'|awk '{printf "%.2f\n",$1}' It returns: 7.435 It should return: 7.44 Any suggestion please? Thanks, Prashant (2 Replies)
Discussion started by: ppat7046
2 Replies

3. Shell Programming and Scripting

awk Division and modulus

I need to read the file divide 3 column with 2nd and run a modulus of 10 and check whether the remainder is zero or not if not print the entire line. cat filename | awk '{ if ($3 / $2 % 10 != 0) print $0}' Whats wrong with it ? (4 Replies)
Discussion started by: dinjo_jo
4 Replies

4. Shell Programming and Scripting

AWK rounding up numbers

Hi, I have managed to round up numbers by using the following command: echo "5.54" | awk '{printf "%.0f\n", $1}' result 6 How can I round up all the numbers in a column in a file and print the lines with the new calculated totals? Thanks, (3 Replies)
Discussion started by: keenboy100
3 Replies

5. UNIX for Advanced & Expert Users

awk: division by zero

I received error "awk: division by zero" while executing the following statement. SunOS 5.10 Generic_142900-15 sun4us sparc FJSV,GPUZC-M echo 8 | awk 'END {printf ("%d\n",NR/$1 + 0.5);}' file1.lst awk: division by zero Can someone provide solution? Thanks Please use code... (11 Replies)
Discussion started by: kumar77
11 Replies

6. Shell Programming and Scripting

awk, floating point and rounding

I had a person bring an interesting problem to me that appears to involve some sort of rounding inside awk. I've verified this with awk and nawk on Solaris as well as with gawk 3.1.5 on a Linux box. The original code fragment he brought me was thus: for (index=0; index < 1; index=index+.1) ... (4 Replies)
Discussion started by: mmyer2
4 Replies

7. Shell Programming and Scripting

awk & division

vmstat|awk '{print $3}'|tail -1 returns 6250511, but what I need is 24416, which is 6250511 divided by 256. Please advise. Thank you so much (2 Replies)
Discussion started by: Daniel Gate
2 Replies

8. Shell Programming and Scripting

awk calculation automatically rounding off the output

I have some calculation in my script which is similar to the below example . I find that sometimes when using large decimal digits, the output gets automatically rounded off and it is affecting the program. I am not able to understand what is happening here.. awk '{ a=6.32498922 a1=6.324... (5 Replies)
Discussion started by: wanderingmind16
5 Replies

9. Shell Programming and Scripting

awk - Division with condition

Hi Friends, I have an input file like this cat input chr1 100 200 1 2 chr1 120 130 na 1 chr1 140 160 1 na chr1 170 180 na na chr1 190 220 0 0 chr1 220 230 nd 1 chr2 330 400 1 nd chr2 410 450 nd nd chr3 500 700 1 1 I want to calculate the division of 4th and 5th columns. But, if... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

10. Shell Programming and Scripting

[awk] rounding a float number?

Heyas Trying to calculate the total size of a file by reading its bitrate. Code snippet: fs_expected() { # # Returns the expected filesize in bytes # pr_str() { ff=$(cat $TMP.info) d="${ff#*bitrate: }" echo "${d%%,*}" | $AWK '{print $1}' | head -n 1 } t_BYTERATE=$((... (9 Replies)
Discussion started by: sea
9 Replies
Num(3o) 							   OCaml library							   Num(3o)

NAME
Num - Operation on arbitrary-precision numbers. Module Module Num Documentation Module Num : sig end Operation on arbitrary-precision numbers. Numbers (type num ) are arbitrary-precision rational numbers, plus the special elements 1/0 (infinity) and 0/0 (undefined). type num = | Int of int | Big_int of Big_int.big_int | Ratio of Ratio.ratio The type of numbers. === Arithmetic operations === val (+/) : num -> num -> num Same as Num.add_num . val add_num : num -> num -> num Addition val minus_num : num -> num Unary negation. val (-/) : num -> num -> num Same as Num.sub_num . val sub_num : num -> num -> num Subtraction val ( */ ) : num -> num -> num Same as Num.mult_num . val mult_num : num -> num -> num Multiplication val square_num : num -> num Squaring val (//) : num -> num -> num Same as Num.div_num . val div_num : num -> num -> num Division val quo_num : num -> num -> num Euclidean division: quotient. val mod_num : num -> num -> num Euclidean division: remainder. val ( **/ ) : num -> num -> num Same as Num.power_num . val power_num : num -> num -> num Exponentiation val abs_num : num -> num Absolute value. val succ_num : num -> num succ n is n+1 val pred_num : num -> num pred n is n-1 val incr_num : num Pervasives.ref -> unit incr r is r:=!r+1 , where r is a reference to a number. val decr_num : num Pervasives.ref -> unit decr r is r:=!r-1 , where r is a reference to a number. val is_integer_num : num -> bool Test if a number is an integer === The four following functions approximate a number by an integer : === val integer_num : num -> num integer_num n returns the integer closest to n . In case of ties, rounds towards zero. val floor_num : num -> num floor_num n returns the largest integer smaller or equal to n . val round_num : num -> num round_num n returns the integer closest to n . In case of ties, rounds off zero. val ceiling_num : num -> num ceiling_num n returns the smallest integer bigger or equal to n . val sign_num : num -> int Return -1 , 0 or 1 according to the sign of the argument. === Comparisons between numbers === val (=/) : num -> num -> bool val (</) : num -> num -> bool val (>/) : num -> num -> bool val (<=/) : num -> num -> bool val (>=/) : num -> num -> bool val (<>/) : num -> num -> bool val eq_num : num -> num -> bool val lt_num : num -> num -> bool val le_num : num -> num -> bool val gt_num : num -> num -> bool val ge_num : num -> num -> bool val compare_num : num -> num -> int Return -1 , 0 or 1 if the first argument is less than, equal to, or greater than the second argument. val max_num : num -> num -> num Return the greater of the two arguments. val min_num : num -> num -> num Return the smaller of the two arguments. === Coercions with strings === val string_of_num : num -> string Convert a number to a string, using fractional notation. val approx_num_fix : int -> num -> string See Num.approx_num_exp . val approx_num_exp : int -> num -> string Approximate a number by a decimal. The first argument is the required precision. The second argument is the number to approximate. Num.approx_num_fix uses decimal notation; the first argument is the number of digits after the decimal point. approx_num_exp uses scien- tific (exponential) notation; the first argument is the number of digits in the mantissa. val num_of_string : string -> num Convert a string to a number. === Coercions between numerical types === val int_of_num : num -> int val num_of_int : int -> num val nat_of_num : num -> Nat.nat val num_of_nat : Nat.nat -> num val num_of_big_int : Big_int.big_int -> num val big_int_of_num : num -> Big_int.big_int val ratio_of_num : num -> Ratio.ratio val num_of_ratio : Ratio.ratio -> num val float_of_num : num -> float OCamldoc 2012-06-26 Num(3o)
All times are GMT -4. The time now is 09:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy