[amibroker] Statistical tests as custom metrics

[Thomas Ludwig <Thomas.Ludwig@xxxxxx>]

Hello,

in "Quantitative Trading Systems" on p. 256, Howard describes a z-score 
test in order to evaluate the statistical significance of a trading 
system. While the formula is easy to write in AFL, I don't think that 
it can be done as a custom metric since the system to be evaluated is 
compared with a Random System. Any idea how to sensibly implement it in 
Amibroker?

I'm using another statistical test proposed by the late Arthur Merrill 
some years ago in S&C. It's the "chi squared with one degree of 
freedom, with the Yates correction". Here's how I implemented it in AB:

//chi squared with one degree of freedom, with the Yates correction
    wi=st.GetValue("WinnersQty");
    Lo=st.GetValue("LosersQty");
    Chi = (abs(wi-Lo)-1)^2/(wi+Lo);
    bo.AddCustomMetric( "Chi-Squared modif.: >10.83: very     
significant(1000:1), >6.64: significant (100:1) , >3.84: probably 
significant (20:1), <3.84: significance doubtful", Chi ); 

While this metric doesn't tell you anything if your system is 
profitable, it tells you if its signals are only pure coincidence 
(simply put). It's remarkable that many systems that seem to be 
promising according to the usual metrics, are below 3.84, i.e. 
significance doubtful. You need either a rather high number of trades 
or a very high percentage of winning trades to shift this metric 
significantly higher. At least for (medium-term) EOD systems (that's 
what I trade) this is not easy to achieve.

What do you think about this metric? Are there other "better" 
statistical metrics? If yes - would you mind sharing the AFL code?

Best regards,

Thomas


Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html
 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/