[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: AmiBroker Measures of Performance



PureBytes Links

Trading Reference Links

So there is no misunderstanding.

I am not posting against Howard - I believe my work is complimentary 
to his.

Also, I am heavily influenced by RalphVince's work and I am not 
adding much, if anything, that is new - I am tinkering around the 
edges of his work and only making it more understandable, if anything.

I am not saying that root cause analysis is the be all and end all.
I am saying that AFAIK it has been almost totally neglected by the 
trading commentators, to this point.

Since it is, at the least, as equally important as equity curve 
(outcomes) analysis I am making a start to redress the imbalance (via 
the UKB) - at least there will be the beginning of a discussion on 
some pragmatic methods to predict the strengths and weaknesses of the 
evaluation metrics currently in vogue.

A couple of specific points:

I agree with Howard that, equity curve metrics that consider risk 
(UlcerIndex etc) are among the better metrics, especially for those 
who are trading on margin (margin calls make us nervous about 
drawdown don't they!).

At the end of the day, while all of the metrics are fine, we all use 
and understand %PA (as reward) and drawdown (as risk) this is our 
first instinctive (habitual) response (funny that because they are 
not actually complimentary metrics).

W/L and PayOff ratio (in it's various formats), are not evaluation 
metrics, as such (not when they stand alone) - they always act in 
tandem and produce a compound result (which is what we are really 
interested in).

I use them as a psychological tool - they focus the mind (we all now 
how difficult an undertaking that is)  on what is important at the 
design stage AND as well as that I am using them to trace the root 
causes of variance in equity curves, which all starts from W/L 
variance and the trade series % variance.

Howard - while you are there I would like to thank you personally for 
sharing and helping developing traders via your book/website and this 
forum. I also thank you for your tolerant attitude - although I am a 
maths lay-person I feel comfortable to discuss my opinions with you.

I can't thank you enough for that since I am always a little nervous 
when talking about maths in front of so many people with maths 
training and experience.

brian_z


--- In amibroker@xxxxxxxxxxxxxxx, "brian_z111" <brian_z111@xxx> wrote:
>
> James,
> 
> > I found it to be lacking in terms of 
> > information regarding metrics.
> 
> IMO Howard didn't set out to write a book on metrics.
> I also believe he was constrained by size limits.
> 
> Over to the discussion:
> 
> Most of the material around only defines the maths - I don't know 
of 
> anyone I could recommend for in depth analysis of metrics - in fact 
> they are all pretty lame and it is a glaring omission from the 
> trading literature.
> 
> The only thing I could do was to figure out the pros and cons for 
> myself.
> 
> IMO most of them are a load of old rubbish - I am interested in 
root 
> causes and follow:
> 
> - trading as a binomial event i.e. Win/Loss +-error and Win%/Loss% 
+- 
> error, which predicts the estimated GeometricMean +- error.
> 
> EGM == arithmetic mean^2 - StDev^2 (from RalphVince - "Mathematics 
of 
> Money Management").
> 
> PowerFactor == (EGM +- StDev)^N is the driver of an equity curve 
> probability cone where the EGM is the reward, and the worstcase eq 
> curve is the risk (you choose your own risk tolerance) - it can be 
> annualised as %.
> 
> From there it is over to RV and MM - everything else is second rate.
> 
> 
> brian_z
> 
> --- In amibroker@xxxxxxxxxxxxxxx, "jamesfarrow2003" 
> <jamesfarrow2003@> wrote:
> >
> > Well, I guess I am still looking.  I just finshed Howard's book
(in 
> > alittle over two days), and while I found it very readable, and 
for 
> > the most part captivating, I found it to be lacking in terms of 
> > information regarding metrics.
> > 
> > I really like the idea he presents about the objective function, 
> but 
> > after basically saying that this is a personal decision, no 
> > information about why you may want to pick RAR vs CAR, or Sharpe 
vs 
> > K-raio, etc, nor anything regarding what kind of values typically 
> > are produced by a "good" system (obviously the definetion of good 
> > will vary based on objective function...kind of a chicken or the 
> egg 
> > question!)
> > 
> > I am glad that I read the appendix, which is where there was a 
> > sample of coding an objective function - very useful.  Also all 
of 
> > the info on walk forward testing and seperation of data is great 
( 
> > and like all good ideas, obvious once you learn about it:)
> > 
> > I would definetly recommend(and already have to sevral co-
workers) 
> > his book to someone looking for methods of system validation, 
> > particularly AmiBroker users.  I am now looking for a book to 
help 
> > me define my personal objective functions!
> > 
> > Another topic I need to research is at what point do you begin 
> > trading a system... after all back testing/optimization/walk 
> > forward/monte carlo is complete, or does it make more sense 
> > to "paper trade" for a while (maybe a set number of trades, or 
> > time)..again probably mostly a personal choice;)
> > 
> > 
> > 
> > --- In amibroker@xxxxxxxxxxxxxxx, Grant Noble <gruntus@> wrote:
> > >
> > > Howard's book is essential for grasping the essence of why any 
> > metric is useful or not. As for 
> > > detailed definitions of each metric just start at the UKB and 
go 
> > from there - Brian is assembling 
> > > loads of link to useful sites.
> > > 
> > > jamesfarrow2003 wrote:
> > > > Can anyone in this group recommend a good book which 
discusses 
> > the 
> > > > various metrics that AmiBroker computes based on backtest 
> > results?
> > > > 
> > > > Thanks,
> > > > 
> > > > james
> > > > 
> > > > 
> > > > ------------------------------------
> > > > 
> > > > Please note that this group is for discussion between users 
> only.
> > > > 
> > > > To get support from AmiBroker please send an e-mail directly 
to 
> > > > SUPPORT {at} amibroker.com
> > > > 
> > > > For NEW RELEASE ANNOUNCEMENTS and other news always check 
> DEVLOG:
> > > > http://www.amibroker.com/devlog/
> > > > 
> > > > For other support material please check also:
> > > > http://www.amibroker.com/support.html
> > > > Yahoo! Groups Links
> > > > 
> > > > 
> > > > 
> > > >
> > >
> >
>



------------------------------------

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/