|
PureBytes Links
Trading Reference Links
|
Hi all!
The performance summary in the backtesting report window contains a
lot of figures. But as useful as these figures are - they do not
really tell us if the backtesting results are statistically
significant.
Therefore I suggest to add a new figure proposed by Arthur Merrill in
Stocks & Commodities some years ago (Bonus Issue 1993). It's called
the "chi squared with one degree of freedom, with the Yates
correction". It's calculated using the following simple formula:
X^2 = ( |R-W| -1)^2 / (R+W)
where: R = number of times right
W = number of times wrong
Interpetation:
Below 3.84: Significance doubtful.
Above 3.84: Probably significant. Probability of one in 20 that the
result was by chance.
Above 6.64: Significant. Probability of one in 100 that the result was
by chance.
Above 10.83: Highly significant. Probability of one in 1,000 that the
result was by chance.
Let`s apply this formula to a simple example. Let's assume that the
backtest gives 10 signals with 7 of them profitable; let's also assume
that the other figures in the backtest report are okay. So the results
look good at the first glance. However, the chi squared figure is just
0.9 - very insignificant!
On the other hand, if the backtest gives 100 signals with 70 of them
profitable the chi squared figure is a whopping 15.21 - highly
significant!
Of course, these are very simple examples and the difference between
them is rather obvious. But that might not be the case in other
examples. And I'm afraid that even obvious things are often
overlooked, especially if the other figures in the backtest report
look good. Therefore in my opinion the chi squared figure would be a
valuable addition for the correct interpretation of the backtest
results.
Tomasz - any chance to add this?
Regards, Thomas
--
Thomas Ludwig
PGP key (RSA) available on request.
Fingerprint: 4356 55FF 3412 277A 9741 1CFA B9A6 7B90
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|