[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: TS calculation problem



PureBytes Links

Trading Reference Links

> Value1 = 1118.274 * 2118.231;
> Value2 = IntPortion(Value1);
> Value3 = FracPortion(Value1);
> 
> This code printed as follows on TS8:
> 1050303   655 2368762.6532939998  2368762.0000000000     0.6532939998  2368762.6532939998

Only because of TS8's higher precision.  On TS2k the calculation 
of value1 doesn't have that many digits of precision.  I ran the 
same code on TS2k and it printed:

1050303  1115         2368763.0000000000          2368762.0000000000                0.7500000000  2368762.7500000000

TS2k only has 6-7 digits of precision in its representation of 
value1.

Gary