|
PureBytes Links
Trading Reference Links
|
Thanks Rajiv,
I have thought that the variance of the chosen metric, from test to test, is an important indicator of the robustness of the system.
I will keep the idea in mind and do some more work on it as I go along.
--- In amibroker@xxxxxxxxxxxxxxx, Rajiv Arya <rajivarya87@xxx> wrote:
>
>
> I like to compute a ratio of the out-sample metric and divide it by the in-sample
>
> metric.
>
> And I like to look for multiple runs of out-sample/in-sample ratio to be above 0.5 and with little fluctuation.
>
>
>
> Rajiv
>
>
>
>
> To: amibroker@xxxxxxxxxxxxxxx
> From: brian_z111@xxx
> Date: Sat, 9 May 2009 01:03:43 +0000
> Subject: [amibroker] Re: Expectancy - and related--specifically K-rato
>
>
>
>
>
>
>
> <snip> The best that I could come up with is that there needs to be a measure of performance and a measure of risk to balance it. There are several such measures in common use and they will each tend to favor different solutions. <snip>
>
> Generally:
>
> - Howard and I disagree on some fundamental points.
> - The best objective function for a single system may not be the best for a system that is going to be included in a portfolio of diversified systems (refer to Howard's post on PortfolioConstruction)
> - If you focus on the derived objective functions you are more likely to see things that aren't there ... the CoreMetrics are more direct and more reliable.
>
> Why?
>
> Because every time you add a factor, with inherent error, to an equation the error propogates and the leveraging effect, of some metrics, then amplifies the error.
> Because the returns are random.
>
> When you are shooting on the range, and firing wide, do you move the target or check your sights? A small offset in the sights produces a large offset when the bullet flies past the target.
>
> --- In amibroker@xxxxxxxxxxxxxxx, "dloyer123" <dloyer123@> wrote:
> >
> > This raises a interesting point.
> >
> > If some objective functions are poor predictors of out of sample performance, are there other objective functions that are good predictors of out of sample performance? Or at least better?
> >
> > It seems like this is the kind of problem where there should be a optimal answer. I wish that I had the math background to find it.
> >
> > The best that I could come up with is that there needs to be a measure of performance and a measure of risk to balance it. There are several such measures in common use and they will each tend to favor different solutions.
> >
> > Since I am not able to pick one on a theoretical basis, I had to resort to empirical testing using Ami's walk forward feature. As a result, I found that net profit performed the worst of all of the objective functions I tried in terms of out of sample profit. In my simple test, UPI performed the best. CAR/MDD also performed well. I don't recall how the others did. I believe that Sharpe was somewhere in the middle and k-ratio didnt do well at all.
> >
> > However, this is just one simple system and I really wish that I had a theoretical basis to hang my hat on it. Results for other systems may be very different and it might be different if I repeated the test now, with a extra year of test data.
> >
> > -Doug
> >
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, Howard B <howardbandy@> wrote:
> >
> > > It is also interesting to note that some objective functions tend to reward
> > > / select values for the logic and parameters that result in trading systems
> > > that do not trade well out-of-sample. In particular, be careful using net
> > > profit, sharpe ratio, and so forth. By all means, define your own custom
> > > objective function metric, program it, and tell AmiBroker to use it.
> > >
> > > Thanks for listening,
> > > Howard
> > >
> > >
> >
>
>
>
>
>
>
>
>
>
> _________________________________________________________________
> Insert movie times and more without leaving Hotmail®.
> http://windowslive.com/Tutorial/Hotmail/QuickAdd?ocid=TXT_TAGLM_WL_HM_Tutorial_QuickAdd1_052009
>
------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|