[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: First calculation problem



PureBytes Links

Trading Reference Links

The ultimate answer to this problem is that you are both sides are right.
Equis chose to use singles because they are the smallest unit of storage and
the fastest for calculations.  Even if they had chosen to use doubles or
long doubles, there are still numbers that cannot be represented.  For
example, the answer Guy is getting might be
12.3999999999999999999999999999999 instead or 12.39999999.  Using BCD is not
a viable option because storage requirements would be way too high and
computation speed would be way too slow.

But there would certainly be fewer of these kinds of errors VISIBLE if
doubles or long doubles were used.

Kent