[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

TS calculation precision problem



PureBytes Links

Trading Reference Links

A few words of warning that I hope will save others from tripping over
similar problems ...

In the past Bob Fulks (and others) have mentioned that the use of
32-bit floating point values in TS2000i and TS6 is a problem.

I didn't fully appreciate this until a few days ago when I tried to
apply the RSquared(14) TS2000i function to a chart of the Emini
Nasdaq. It's a builtin function and I wasn't using a very long length
so it should work, right?  Wrong.

The values returned by the TS2000i RSquared() function only barely
resembled the correct values.  After digging into it, I found that the
intermediate values in the RSquared computation were large enough that
the 32-bit floats were unable to retain enough precision for the end
result to be useful.

In order to get an RSquared function that calculated correctly, I had
to write my own as a DLL function that uses 64-bit doubles for the
intermediate results of the calculation.

Rich