|
PureBytes Links
Trading Reference Links
|
Mike,
thanks a lot for these comments. I will certainly look into your posts!
Best regards,
Thomas
> Hi,
>
> The quickest way to do it might be to run your backtest, output the
> trade list to .csv file for loading into Excel, then do the same
> for a Random strategy. You could then use Excel to perform a t-test
> (equivalent to z-test after sufficient data points) as follows:
>
> - Add the "Analysis TookPak" to Excel via "Tools | Add ins..." menu.
> - Run "Tools | Data Analysis..." menu
> - Select "t-Test: Two variables assuming equal variances".
> - Select the trade percentage return column from each of the strategy
> results for the variable ranges.
> - Enter zero as the hypothesized mean difference
> - Indicate where you want the resulting table to get placed
> - Click OK
>
> In the resulting table, if you see P(T<=t) one tail with a value
> of .05 or less, then you've got statistical significance with 95%
> confidence.
>
> Along these same lines, I recently posted comments (including code)
> for a similar approach based on applying your strategy signals
> against detrended daily market change data (David Aronson's book).
> The end result is the same; comparing your strategy against random
> returns.
>
> The first script (message 120676) will detrend the average daily
> change of the market such that the average change will be zero. This
> implies that if we were to randomly apply signals for long and short
> against the data, the return would be zero.
>
> The second script (message 120718) gives an example "always in"
> strategy that applies its market signals against the detrended daily
> changes.
>
> You would then run the t-test between the means of the two sets of
> results (i.e the zero mean of the detrended daily change against the
> presumably non zero mean of your strategy - as applied against the
> detrended daily change).
>
> To prepare the results for export to Excel, add the following lines
> to the end of the first script (as opposed to having to cut and paste
> from the Quote Editor of the resulting composite):
>
> SetForeign("~" + Name());
> Filter = NOT IsNan(Close);
> AddColumn(Close, "Detrended Ln", 10.9);
> RestorePriceArrays();
>
> Similarly, add these lines to the bottom of the second script:
>
> SetForeign("~" + Name() + "Return");
> Filter = NOT IsNan(Close);
> AddColumn(Close, "Detrended Result", 10.9);
> RestorePriceArrays();
>
> Now after running the first script as an exploration against your
> symbol, you can just export the result to .csv and load into Excel.
>
> Same for running the second script as an exploration against that
> same symbol. Then perform t-test as above using the two output
> columns as the variable ranges.
>
> Note that the values are logarithms of the percent daily change, so
> don't try interpreting them as price values.
>
> Mike
>
>
> --- In amibroker@xxxxxxxxxxxxxxx, Thomas Ludwig <Thomas.Ludwig@xxx>
>
> wrote:
> > Hello,
> >
> > in "Quantitative Trading Systems" on p. 256, Howard describes a z-
>
> score
>
> > test in order to evaluate the statistical significance of a trading
> > system. While the formula is easy to write in AFL, I don't think
>
> that
>
> > it can be done as a custom metric since the system to be evaluated
>
> is
>
> > compared with a Random System. Any idea how to sensibly implement
>
> it in
>
> > Amibroker?
> >
> > I'm using another statistical test proposed by the late Arthur
>
> Merrill
>
> > some years ago in S&C. It's the "chi squared with one degree of
> > freedom, with the Yates correction". Here's how I implemented it in
>
> AB:
> > //chi squared with one degree of freedom, with the Yates correction
> > wi=st.GetValue("WinnersQty");
> > Lo=st.GetValue("LosersQty");
> > Chi = (abs(wi-Lo)-1)^2/(wi+Lo);
> > bo.AddCustomMetric( "Chi-Squared modif.: >10.83: very
> > significant(1000:1), >6.64: significant (100:1) , >3.84: probably
> > significant (20:1), <3.84: significance doubtful", Chi );
> >
> > While this metric doesn't tell you anything if your system is
> > profitable, it tells you if its signals are only pure coincidence
> > (simply put). It's remarkable that many systems that seem to be
> > promising according to the usual metrics, are below 3.84, i.e.
> > significance doubtful. You need either a rather high number of
>
> trades
>
> > or a very high percentage of winning trades to shift this metric
> > significantly higher. At least for (medium-term) EOD systems
>
> (that's
>
> > what I trade) this is not easy to achieve.
> >
> > What do you think about this metric? Are there other "better"
> > statistical metrics? If yes - would you mind sharing the AFL code?
> >
> > Best regards,
> >
> > Thomas
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|