Sponsored Content
Top Forums UNIX for Beginners Questions & Answers awk division without rounding Post 303046061 by RudiC on Wednesday 22nd of April 2020 03:46:27 PM
Old 04-22-2020
Decimal numbers represented by a binary system are ALWAYS rounded / subject to rounding. For example, your first number, given as 0.123456789012, is internally approximated / stored as 0.123456789011999995553381381796, which itself is a rounded value already. You may end up less than or greater than your target value, or, in rare cases, exactly on target (0.5, or 0.25, for example).

So, your request is a bit diffucult to fulfill. You can try awk's OFMT variable set to a fixed output field length, but, surprise, additional undesired decimal places occur:
Code:
awk '{val = $1 / 100; print val}' OFMT="%.20f" file
0.00123456789012000006
0.00123456789012344999
0.00123456789012345671


If that is unsatisfying, you can resort to formatted print and a tricky taylored field length computation:

Code:
awk '{val = $1 / 100; printf "%.*f\n", length($1), val}' file
0.00123456789012
0.00123456789012345
0.0012345678901234567

This works as there are two additional places needed for the division by 100, but length ($1) already counts two characters: the leading 0 and dot. If you have more significant leading digits, additional measures / steps need to be taken.
This User Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Annoying rounding issue in awk

Hello I am getting this very annoying issue in awk: awk '{a=12825;b=a*1.25; print b}' test 16031.2 Thing is the multiplication result is wrong... Result should be 16031.25. I think the issue only happens on bigger numbers. What can I do to get passed this? Thanks by advance (3 Replies)
Discussion started by: Indalecio
3 Replies

2. Shell Programming and Scripting

Rounding issue with awk

Hi Friends, I am trying to round following number. 0.07435000 echo "0.07435000"|awk '{printf "%s\n",$1*100}'|awk '{printf "%.2f\n",$1}' It returns: 7.435 It should return: 7.44 Any suggestion please? Thanks, Prashant (2 Replies)
Discussion started by: ppat7046
2 Replies

3. Shell Programming and Scripting

awk Division and modulus

I need to read the file divide 3 column with 2nd and run a modulus of 10 and check whether the remainder is zero or not if not print the entire line. cat filename | awk '{ if ($3 / $2 % 10 != 0) print $0}' Whats wrong with it ? (4 Replies)
Discussion started by: dinjo_jo
4 Replies

4. Shell Programming and Scripting

AWK rounding up numbers

Hi, I have managed to round up numbers by using the following command: echo "5.54" | awk '{printf "%.0f\n", $1}' result 6 How can I round up all the numbers in a column in a file and print the lines with the new calculated totals? Thanks, (3 Replies)
Discussion started by: keenboy100
3 Replies

5. UNIX for Advanced & Expert Users

awk: division by zero

I received error "awk: division by zero" while executing the following statement. SunOS 5.10 Generic_142900-15 sun4us sparc FJSV,GPUZC-M echo 8 | awk 'END {printf ("%d\n",NR/$1 + 0.5);}' file1.lst awk: division by zero Can someone provide solution? Thanks Please use code... (11 Replies)
Discussion started by: kumar77
11 Replies

6. Shell Programming and Scripting

awk, floating point and rounding

I had a person bring an interesting problem to me that appears to involve some sort of rounding inside awk. I've verified this with awk and nawk on Solaris as well as with gawk 3.1.5 on a Linux box. The original code fragment he brought me was thus: for (index=0; index < 1; index=index+.1) ... (4 Replies)
Discussion started by: mmyer2
4 Replies

7. Shell Programming and Scripting

awk & division

vmstat|awk '{print $3}'|tail -1 returns 6250511, but what I need is 24416, which is 6250511 divided by 256. Please advise. Thank you so much (2 Replies)
Discussion started by: Daniel Gate
2 Replies

8. Shell Programming and Scripting

awk calculation automatically rounding off the output

I have some calculation in my script which is similar to the below example . I find that sometimes when using large decimal digits, the output gets automatically rounded off and it is affecting the program. I am not able to understand what is happening here.. awk '{ a=6.32498922 a1=6.324... (5 Replies)
Discussion started by: wanderingmind16
5 Replies

9. Shell Programming and Scripting

awk - Division with condition

Hi Friends, I have an input file like this cat input chr1 100 200 1 2 chr1 120 130 na 1 chr1 140 160 1 na chr1 170 180 na na chr1 190 220 0 0 chr1 220 230 nd 1 chr2 330 400 1 nd chr2 410 450 nd nd chr3 500 700 1 1 I want to calculate the division of 4th and 5th columns. But, if... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

10. Shell Programming and Scripting

[awk] rounding a float number?

Heyas Trying to calculate the total size of a file by reading its bitrate. Code snippet: fs_expected() { # # Returns the expected filesize in bytes # pr_str() { ff=$(cat $TMP.info) d="${ff#*bitrate: }" echo "${d%%,*}" | $AWK '{print $1}' | head -n 1 } t_BYTERATE=$((... (9 Replies)
Discussion started by: sea
9 Replies
SNPRINTB(3)						   BSD Library Functions Manual 					       SNPRINTB(3)

NAME
snprintb -- bitmask output conversion LIBRARY
System Utilities Library (libutil, -lutil) SYNOPSIS
#include <util.h> int snprintb(char *buf, size_t buflen, const char *fmt, uint64_t val); int snprintb_m(char *buf, size_t buflen, const char *fmt, uint64_t val, size_t max); DESCRIPTION
The snprintb() function formats a bitmask into a mnemonic form suitable for printing. This conversion is useful for decoding bit fields in device registers. It formats the integer val into the buffer buf, of size buflen, using a specified radix and an interpretation of the bits within that integer as though they were flags. The buffer is always NUL-terminated. If the buffer buf is too small to hold the formatted output, snprintb() will fill as much as it can, and return the number of bytes that would have written if the buffer was long enough excluding the terminating NUL. The decoding directive string fmt describes how the bitfield is to be interpreted and displayed. It follows two possible syntaxes, referred to as ``old'' and ``new''. The main advantage of the ``new'' formatting is that it is capable of handling multi-bit fields. The first character of fmt may be 177, indicating that the remainder of the format string follows the ``new'' syntax. The second character (the first for the old format) is a binary character representation of the output numeral base in which the bitfield will be printed before it is decoded. Recognized radix values (in C escape-character format) are 10 (octal), 12 (decimal), and 20 (hexadecimal). The remaining characters in fmt are interpreted as a list of bit-position-description pairs. From here the syntaxes diverge. The ``old'' format syntax is series of bit-position-description pairs. Each begins with a binary character value that represents the posi- tion of the bit being described. A bit position value of one describes the least significant bit. Whereas a position value of 32 (octal 40, hexadecimal 20, the ASCII space character) describes the most significant bit. The remaining characters in a bit-position-description pair are the characters to print should the bit being described be set. Description strings are delimited by the next bit position value character encountered (distinguishable by its value being <= 32), or the end of the decoding directive string itself. For the ``new'' format syntax, a bit-position-description begins with a field type followed by a binary bit-position and possibly a field length. The least significant bit is bit-position zero, unlike the ``old'' syntax where it is one. bB Describes a bit position. The bit-position B indicates the corresponding bit, as in the ``old'' format. fBL Describes a multi-bit field beginning at bit-position B and having a bit-length of L. The remaining characters are printed as a description of the field followed by '=' and the value of the field. The value of the field is printed in the base specified as the second character of the decoding directive string fmt. FBL Describes a multi-bit field like 'f', but just extracts the value for use with the '=' and ':' formatting directives described below. =V The field previously extracted by the last 'f' or 'F' operator is compared to the byte 'V' (for values 0 through 255). If they are equal, '=' followed by the string following 'V' is printed. This and the ':' operator may be repeated to annotate multiple possible values. :V Operates like the '=' operator, but omits the leading '='. Finally, each field is delimited by a NUL ('') character. By convention, the format string has an additional NUL character at the end, following that delimiting the last bit-position-description pair. The snprintb_m() function accepts an additional max argument. If this argument is zero, the snprintb_m() function returns exactly the same results in the buf as the snprintb() function. If the max argument is present and has a non-zero value, it represents the maximum length of a formatted string. If the formatted string would require more than max characters, the snprintb_m() function returns multiple formatted strings in the output buffer buf. Each string is NUL-terminated, and the last string is followed by an additional NUL character (or, if you prefer, a zero-length string). RETURN VALUES
The snprintb() and snprintb_m() functions return the number of bytes that would have written to the buffer if there was adequate space, excluding the final terminating NUL, or -1 in case an error occurred. For snprintb_m(), the NUL characters terminating each individual string are included in the total number of bytes. EXAMPLES
Two examples of the old formatting style: snprintb(buf, buflen, "102BITTWO1BITONE", 3) => "3<BITTWO,BITONE>" snprintb(buf, buflen, "20x10NOTBOOTx0fFPPx0eSDVMAx0cVIDEO" "x0bLORESx0aFPAx09DIAGx07CACHE" "x06IOCACHEx05LOOPBACKx04DBGCACHE", 0xe860) => "e860<NOTBOOT,FPP,SDVMA,VIDEO,CACHE,IOCACHE>" An example of the new formatting style: snprintb(buf, buflen, "17720bLSBb1_BITONEf44NIBBLE2" "fx104BURST=4FOUR=xfSIXTEEN" "bx1fMSB", 0x800f0701) => "800f0701<LSB,NIBBLE2=0,BURST=f=SIXTEEN,MSB>" ERRORS
snprintb() will fail if: [EINVAL] The leading character does not describe a supported format, or snprintf() failed. SEE ALSO
printf(3), snprintf(3) HISTORY
The snprintb() function was originally implemented as a non-standard %b format string for the kernel printf() function in NetBSD 1.5 and ear- lier releases. It was called bitmask_snprintf() in NetBSD 5.0 and earlier releases. AUTHORS
The ``new'' format was the invention of Chris Torek. BSD
May 7, 2009 BSD
All times are GMT -4. The time now is 08:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy