|
PureBytes Links
Trading Reference Links
|
Hi,
The quickest way to do it might be to run your backtest, output the
trade list to .csv file for loading into Excel, then do the same
for a Random strategy. You could then use Excel to perform a t-test
(equivalent to z-test after sufficient data points) as follows:
- Add the "Analysis TookPak" to Excel via "Tools | Add ins..." menu.
- Run "Tools | Data Analysis..." menu
- Select "t-Test: Two variables assuming equal variances".
- Select the trade percentage return column from each of the strategy
results for the variable ranges.
- Enter zero as the hypothesized mean difference
- Indicate where you want the resulting table to get placed
- Click OK
In the resulting table, if you see P(T<=t) one tail with a value
of .05 or less, then you've got statistical significance with 95%
confidence.
Along these same lines, I recently posted comments (including code)
for a similar approach based on applying your strategy signals
against detrended daily market change data (David Aronson's book).
The end result is the same; comparing your strategy against random
returns.
The first script (message 120676) will detrend the average daily
change of the market such that the average change will be zero. This
implies that if we were to randomly apply signals for long and short
against the data, the return would be zero.
The second script (message 120718) gives an example "always in"
strategy that applies its market signals against the detrended daily
changes.
You would then run the t-test between the means of the two sets of
results (i.e the zero mean of the detrended daily change against the
presumably non zero mean of your strategy - as applied against the
detrended daily change).
To prepare the results for export to Excel, add the following lines
to the end of the first script (as opposed to having to cut and paste
from the Quote Editor of the resulting composite):
SetForeign("~" + Name());
Filter = NOT IsNan(Close);
AddColumn(Close, "Detrended Ln", 10.9);
RestorePriceArrays();
Similarly, add these lines to the bottom of the second script:
SetForeign("~" + Name() + "Return");
Filter = NOT IsNan(Close);
AddColumn(Close, "Detrended Result", 10.9);
RestorePriceArrays();
Now after running the first script as an exploration against your
symbol, you can just export the result to .csv and load into Excel.
Same for running the second script as an exploration against that
same symbol. Then perform t-test as above using the two output
columns as the variable ranges.
Note that the values are logarithms of the percent daily change, so
don't try interpreting them as price values.
Mike
--- In amibroker@xxxxxxxxxxxxxxx, Thomas Ludwig <Thomas.Ludwig@xxx>
wrote:
>
> Hello,
>
> in "Quantitative Trading Systems" on p. 256, Howard describes a z-
score
> test in order to evaluate the statistical significance of a trading
> system. While the formula is easy to write in AFL, I don't think
that
> it can be done as a custom metric since the system to be evaluated
is
> compared with a Random System. Any idea how to sensibly implement
it in
> Amibroker?
>
> I'm using another statistical test proposed by the late Arthur
Merrill
> some years ago in S&C. It's the "chi squared with one degree of
> freedom, with the Yates correction". Here's how I implemented it in
AB:
>
> //chi squared with one degree of freedom, with the Yates correction
> wi=st.GetValue("WinnersQty");
> Lo=st.GetValue("LosersQty");
> Chi = (abs(wi-Lo)-1)^2/(wi+Lo);
> bo.AddCustomMetric( "Chi-Squared modif.: >10.83: very
> significant(1000:1), >6.64: significant (100:1) , >3.84: probably
> significant (20:1), <3.84: significance doubtful", Chi );
>
> While this metric doesn't tell you anything if your system is
> profitable, it tells you if its signals are only pure coincidence
> (simply put). It's remarkable that many systems that seem to be
> promising according to the usual metrics, are below 3.84, i.e.
> significance doubtful. You need either a rather high number of
trades
> or a very high percentage of winning trades to shift this metric
> significantly higher. At least for (medium-term) EOD systems
(that's
> what I trade) this is not easy to achieve.
>
> What do you think about this metric? Are there other "better"
> statistical metrics? If yes - would you mind sharing the AFL code?
>
> Best regards,
>
> Thomas
>
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|