|
PureBytes Links
Trading Reference Links
|
If Tomasz ever did add this, it would much more useful for the report
to contain the p values rather than raw chi-square. Only problem is,
with any large backtest you'll get a lot of "significant" results by
chance alone. Then you'd need another test, lol, to see if you have
more significant results than you'd expect by chance given the size of
the backtest. I'm a huge fan of Arthur Merrill but can think of more
useful (statistical) functionality to add (in my opinion) if Tomasz
were ever so inclined. BTW, this is very easy to do in a spreadsheet
with results pasted from AA. You could even make a template so it
would be nearly effortless.
--- In amibroker@xxxxxxxxxxxxxxx, Thomas Ludwig <Thomas.Ludwig@xxxx>
wrote:
> Hi all!
> The performance summary in the backtesting report window contains a
> lot of figures. But as useful as these figures are - they do not
> really tell us if the backtesting results are statistically
> significant.
>
> Therefore I suggest to add a new figure proposed by Arthur Merrill
in
> Stocks & Commodities some years ago (Bonus Issue 1993). It's called
> the "chi squared with one degree of freedom, with the Yates
> correction". It's calculated using the following simple formula:
>
> X^2 = ( |R-W| -1)^2 / (R+W)
> where: R = number of times right
> W = number of times wrong
>
> Interpetation:
> Below 3.84: Significance doubtful.
> Above 3.84: Probably significant. Probability of one in 20 that the
> result was by chance.
> Above 6.64: Significant. Probability of one in 100 that the result
was
> by chance.
> Above 10.83: Highly significant. Probability of one in 1,000 that
the
> result was by chance.
>
> Let`s apply this formula to a simple example. Let's assume that the
> backtest gives 10 signals with 7 of them profitable; let's also
assume
> that the other figures in the backtest report are okay. So the
results
> look good at the first glance. However, the chi squared figure is
just
> 0.9 - very insignificant!
> On the other hand, if the backtest gives 100 signals with 70 of them
> profitable the chi squared figure is a whopping 15.21 - highly
> significant!
>
> Of course, these are very simple examples and the difference between
> them is rather obvious. But that might not be the case in other
> examples. And I'm afraid that even obvious things are often
> overlooked, especially if the other figures in the backtest report
> look good. Therefore in my opinion the chi squared figure would be
a
> valuable addition for the correct interpretation of the backtest
> results.
>
> Tomasz - any chance to add this?
>
> Regards, Thomas
> --
> Thomas Ludwig
> PGP key (RSA) available on request.
> Fingerprint: 4356 55FF 3412 277A 9741 1CFA B9A6 7B90
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|