|
PureBytes Links
Trading Reference Links
|
Sent privately:
>Whoops Bob, I believe you are wrong on that last one....
>the single prec floating spec has a limit of
>+/-2,147,000,000 or something like that.....
>effectively 9 digits of precision....give or take.
>Pierre suggested 7 9's.....when in fact 9 9's would have been the correct
>test of the limit.
I was going from a post by Gary Fritz to the Code List a while back.
It seems correct to me... See attached.
I did some checking and Gary is correct. The IEEE floating point
format is described in detail at:
<http://www.psc.edu/general/software/packages/ieee/ieee.html>
But in any case Pierre's test is for an absolute value when, in fact,
the exponent can be as high as 2^128 which about 3.40 * 10^38 (a very
large number).
Bob
At 10:02 AM -0600 7/2/01, Gary Fritz wrote (Code List):
>As background: single-precision floats have a 24-bit mantissa. That
>is, they use 24 bits to represent the base of the number, and 8 bits
>for the exponent. 24 bits can represent a signed value in the range
>+/-8388608 or so, or not quite 7 digits of precision. You can
>calculate the decimal (base 10) digits available with the formula
>
> (MatissaBits - 1) * log10(2)
>
>MantissaBits is 24, and we must subtract one bit for the sign of the
>mantissa. That formula says there are 6.92 decimal digits available.
|