Sponsored Content
Full Discussion: precision getting?
Top Forums UNIX for Dummies Questions & Answers precision getting? Post 8344 by Perderabo on Wednesday 10th of October 2001 10:26:49 AM
Old 10-10-2001
Sun uses IEEE standard floating point numbers. Standard Precision is 32 bits and double precision is 64 bits. There is also long double which is 128 bits on a sparc but only 80 bits if solaris is running on an intel.

Click here for some more details.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

bc arbitrary precision cal lang

Hello all, I don't know ksh that well but when I try to run a simple script using the bc command I get "unexpected '.' ", has any one encouter this or help me. I'm running under redaht 7.2. Here is a simple example: #!/bin/ksh # kilo=`echo "2495 * .25"|bc` if then echo "LESS THEN"... (2 Replies)
Discussion started by: larry
2 Replies

2. HP-UX

GMP multi precision math library

Hello all, I'm having trouble building the GNU Multi Precision (GMP 4.1.3) on the HP Intel Itanium 2 HP-UX 11i I'm hoping someone out there has had some experience building GMP on this platform. After running the ./configure and doing a make I get an error saying: `.rodata` is not a section ... (3 Replies)
Discussion started by: vyl
3 Replies

3. Programming

max precision in float

Hi, As all of us know that size of int is machine as well as compiler dependent we can get the range and size of an int , float and char etc in /usr/include/limits.h header file of a compiler but could any one tell me how to get info of precision of float and ... (6 Replies)
Discussion started by: useless79
6 Replies

4. Shell Programming and Scripting

unix diff with numerical precision

Hi, I search a unix utility to diff two files, that takes as argument numerical precision as well. If two files are different only in numbers and the difference is less than precision then the result as they are the same. The precision may be absolute (x1-x2) or relative (x2/x1). Also I need the... (1 Reply)
Discussion started by: stella
1 Replies

5. Shell Programming and Scripting

Count the decimal numbers with 6 precision

Hi guys, I have a single column file with millions of records. I want to count the number of records with 6 decimal precision. for ex: 1234.12 1234.132 12345.12345 1234.1 1234.13 1234.123456 243435.454555 i need to count the number of records with precision of 6 ( i.e here the... (4 Replies)
Discussion started by: marcus_kosaman
4 Replies

6. Shell Programming and Scripting

Double-precision result (awk)

hi, I am a new awk user, now i want to change my result from 1 ca 0.2057422D-01 -0.7179106D-02 -0.5600872D-02 2 o 0.2463722D-01 -0.1554542D-01 0.3110649D-01 3 h -0.1068047D-01 0.1016889D-01 -0.4088230D-02 to 1 ca 0.02057422 -0.007179106 -0.005600872 2 o 0.02463722 -0.01554542 ... (4 Replies)
Discussion started by: wanchem
4 Replies

7. Shell Programming and Scripting

change precision of bash variable

I have a variable in a bash script, DISTANCE=`awk 'BEGIN {FS="\t"} {if (NR==2) print $3;}' $OUTFILE` this is a real number taken from a file. The value are like, 0.334561754018 I am using this value in a file name, '$NAME'_'$DISTANCE'.txt I would like to shorten the number some to... (4 Replies)
Discussion started by: LMHmedchem
4 Replies

8. Shell Programming and Scripting

Get timestamp with millisecond precision

Hi All, could any body let me know. how to get timestamp with millisecond precision in unix bash shell. example -->2005-12-06- 4-22-35-195 please help me. Thanks, Krupa:wall: (3 Replies)
Discussion started by: krupasindhu18
3 Replies

9. Shell Programming and Scripting

Precision in print

I have a shell script which prints a variable to a text file (sorry if this is the wrong terminology, I'm new to this): awk `{sumdiffsqt+=$1;stdevt=sqrt(sumdiffsqt/('${n}'-1));errt=stdevt/sqrt('${n}')} END {print '$period',stdevt,errt}' diffsqt.txt >> ${xstation1}_${xstation2}_tt.err I need... (4 Replies)
Discussion started by: claire.a
4 Replies

10. How to Post in the The UNIX and Linux Forums

How to get defined precision after arithmetic operation using syncsort?

I have to do some arithmetic operation on Field 8 which is calculated by Field 9/Field 7 Suppose i have data like : 0800123456|JAN|2017|JAN|2018|0800123456|0|0.0000|0.00| 0800234567|JAN|2017|JAN|2018|0800234567|4|2.5812|10.32| 0800666666|JAN|2017|JAN|2018|0800666666|2|1.7255|3.45|... (0 Replies)
Discussion started by: pumrao
0 Replies
values(5)							File Formats Manual							 values(5)

NAME
values - machine-dependent values SYNOPSIS
DESCRIPTION
This file contains a set of manifest constants, conditionally defined for particular processor architectures. The model assumed for integers is binary representation (one's or two's complement), where the sign is represented by the value of the high-order bit. The number of bits in a specified type (e.g., int). The value of a short integer with only the high-order bit set (in most implementations, 0x8000). The value of a long integer with only the high-order bit set (in most implementations, 0x80000000). The value of a regular integer with only the high-order bit set (usually the same as or The maximum value of a signed short integer (in most implementations, 0x7FFF == 32767). The maximum value of a signed long integer (in most implementations, 0x7FFFFFFF == 2147483647). The maximum value of a signed regular integer (usually the same as or The maximum value of a single-precision floating-point number, and its natural logarithm. The maximum value of a double-precision floating-point number, and its natural logarithm. The minimum positive value of a single-precision floating-point number, and its natural logarithm. The minimum positive value of a double-precision floating-point number, and its natural logarithm. The number of significant bits in the mantissa of a single-precision floating-point number. The number of significant bits in the mantissa of a double-precision floating-point number. FILES
SEE ALSO
intro(3), math(5). STANDARDS CONFORMANCE
values(5)
All times are GMT -4. The time now is 02:02 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy