[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: AmiBroker Measures of Performance



PureBytes Links

Trading Reference Links

Howard, Brian, and Dennis,

Thanks for your thoughtful replies.  This truely is a good 
discussion group, especially when the author of the book under 
dicsussion joins in!

I will definetly take Dennis's advice about begining trading a new 
system with small capital at stake (with the understanding that it 
must be large enough to overcome fees and slippage).

When I mentioned the "chicken or the egg", I did not mean to infer 
that the objective function was not a personal choice, but rather 
how does a new trader come up with acceptable metrics(ie a personal 
choice) without a decent understanding of the available metrics...
To phase this differently, in Howards post, he mentioned several 
types of traders (ie want high return regardless of risk, want 
higher return, but with low drawdown), it may be useful for a new 
trader to be able to have reference material that says if you are 
type I, you would most likely use CAR, if youy are type II you would 
likely use CAR/MaxDD or RAR/MAxDD, etc.  This would still force the 
individual trader to perform some self analysis as to what type of 
trader he/she is, but would help facilitate the creation of the 
objective function.

I actually did enjoy seeing the plots at the end of Chapter 15, 
which showed some equity curves vs metrics.  I have already begun 
the process you identified, but it is somewhat difficult to effect a 
change to a system to increase a given metric to give a series of 
charts to inspect.  

I also appeciated Howard comment about the performance of various 
metrics during the in-sample and out-of-sample periods.  If these 
are generally true, that is great information to have as I form my 
objective function!

Brian is right that there seems to be a lack of information on this 
topic - I spent over an hour paging through books at my local 
bookstore this evening with no luck.

BTW, I have also placed my order for one of Ralph Vinces book, per 
Brians recommendation (and his mention in Howards book) to help me 
learn more about position sizing. 

Again, thanks for taking time out to have a discussion with 
a "newbie".  Cheers!

--- In amibroker@xxxxxxxxxxxxxxx, "brian_z111" <brian_z111@xxx> 
wrote:
>
> So there is no misunderstanding.
> 
> I am not posting against Howard - I believe my work is 
complimentary 
> to his.
> 
> Also, I am heavily influenced by RalphVince's work and I am not 
> adding much, if anything, that is new - I am tinkering around the 
> edges of his work and only making it more understandable, if 
anything.
> 
> I am not saying that root cause analysis is the be all and end all.
> I am saying that AFAIK it has been almost totally neglected by the 
> trading commentators, to this point.
> 
> Since it is, at the least, as equally important as equity curve 
> (outcomes) analysis I am making a start to redress the imbalance 
(via 
> the UKB) - at least there will be the beginning of a discussion on 
> some pragmatic methods to predict the strengths and weaknesses of 
the 
> evaluation metrics currently in vogue.
> 
> A couple of specific points:
> 
> I agree with Howard that, equity curve metrics that consider risk 
> (UlcerIndex etc) are among the better metrics, especially for 
those 
> who are trading on margin (margin calls make us nervous about 
> drawdown don't they!).
> 
> At the end of the day, while all of the metrics are fine, we all 
use 
> and understand %PA (as reward) and drawdown (as risk) this is our 
> first instinctive (habitual) response (funny that because they are 
> not actually complimentary metrics).
> 
> W/L and PayOff ratio (in it's various formats), are not evaluation 
> metrics, as such (not when they stand alone) - they always act in 
> tandem and produce a compound result (which is what we are really 
> interested in).
> 
> I use them as a psychological tool - they focus the mind (we all 
now 
> how difficult an undertaking that is)  on what is important at the 
> design stage AND as well as that I am using them to trace the root 
> causes of variance in equity curves, which all starts from W/L 
> variance and the trade series % variance.
> 
> Howard - while you are there I would like to thank you personally 
for 
> sharing and helping developing traders via your book/website and 
this 
> forum. I also thank you for your tolerant attitude - although I am 
a 
> maths lay-person I feel comfortable to discuss my opinions with 
you.
> 
> I can't thank you enough for that since I am always a little 
nervous 
> when talking about maths in front of so many people with maths 
> training and experience.
> 
> brian_z
> 
> 
> --- In amibroker@xxxxxxxxxxxxxxx, "brian_z111" <brian_z111@> wrote:
> >
> > James,
> > 
> > > I found it to be lacking in terms of 
> > > information regarding metrics.
> > 
> > IMO Howard didn't set out to write a book on metrics.
> > I also believe he was constrained by size limits.
> > 
> > Over to the discussion:
> > 
> > Most of the material around only defines the maths - I don't 
know 
> of 
> > anyone I could recommend for in depth analysis of metrics - in 
fact 
> > they are all pretty lame and it is a glaring omission from the 
> > trading literature.
> > 
> > The only thing I could do was to figure out the pros and cons 
for 
> > myself.
> > 
> > IMO most of them are a load of old rubbish - I am interested in 
> root 
> > causes and follow:
> > 
> > - trading as a binomial event i.e. Win/Loss +-error and 
Win%/Loss% 
> +- 
> > error, which predicts the estimated GeometricMean +- error.
> > 
> > EGM == arithmetic mean^2 - StDev^2 (from RalphVince -
 "Mathematics 
> of 
> > Money Management").
> > 
> > PowerFactor == (EGM +- StDev)^N is the driver of an equity curve 
> > probability cone where the EGM is the reward, and the worstcase 
eq 
> > curve is the risk (you choose your own risk tolerance) - it can 
be 
> > annualised as %.
> > 
> > From there it is over to RV and MM - everything else is second 
rate.
> > 
> > 
> > brian_z
> > 
> > --- In amibroker@xxxxxxxxxxxxxxx, "jamesfarrow2003" 
> > <jamesfarrow2003@> wrote:
> > >
> > > Well, I guess I am still looking.  I just finshed Howard's book
> (in 
> > > alittle over two days), and while I found it very readable, 
and 
> for 
> > > the most part captivating, I found it to be lacking in terms 
of 
> > > information regarding metrics.
> > > 
> > > I really like the idea he presents about the objective 
function, 
> > but 
> > > after basically saying that this is a personal decision, no 
> > > information about why you may want to pick RAR vs CAR, or 
Sharpe 
> vs 
> > > K-raio, etc, nor anything regarding what kind of values 
typically 
> > > are produced by a "good" system (obviously the definetion of 
good 
> > > will vary based on objective function...kind of a chicken or 
the 
> > egg 
> > > question!)
> > > 
> > > I am glad that I read the appendix, which is where there was a 
> > > sample of coding an objective function - very useful.  Also 
all 
> of 
> > > the info on walk forward testing and seperation of data is 
great 
> ( 
> > > and like all good ideas, obvious once you learn about it:)
> > > 
> > > I would definetly recommend(and already have to sevral co-
> workers) 
> > > his book to someone looking for methods of system validation, 
> > > particularly AmiBroker users.  I am now looking for a book to 
> help 
> > > me define my personal objective functions!
> > > 
> > > Another topic I need to research is at what point do you begin 
> > > trading a system... after all back testing/optimization/walk 
> > > forward/monte carlo is complete, or does it make more sense 
> > > to "paper trade" for a while (maybe a set number of trades, or 
> > > time)..again probably mostly a personal choice;)
> > > 
> > > 
> > > 
> > > --- In amibroker@xxxxxxxxxxxxxxx, Grant Noble <gruntus@> wrote:
> > > >
> > > > Howard's book is essential for grasping the essence of why 
any 
> > > metric is useful or not. As for 
> > > > detailed definitions of each metric just start at the UKB 
and 
> go 
> > > from there - Brian is assembling 
> > > > loads of link to useful sites.
> > > > 
> > > > jamesfarrow2003 wrote:
> > > > > Can anyone in this group recommend a good book which 
> discusses 
> > > the 
> > > > > various metrics that AmiBroker computes based on backtest 
> > > results?
> > > > > 
> > > > > Thanks,
> > > > > 
> > > > > james
> > > > > 
> > > > > 
> > > > > ------------------------------------
> > > > > 
> > > > > Please note that this group is for discussion between 
users 
> > only.
> > > > > 
> > > > > To get support from AmiBroker please send an e-mail 
directly 
> to 
> > > > > SUPPORT {at} amibroker.com
> > > > > 
> > > > > For NEW RELEASE ANNOUNCEMENTS and other news always check 
> > DEVLOG:
> > > > > http://www.amibroker.com/devlog/
> > > > > 
> > > > > For other support material please check also:
> > > > > http://www.amibroker.com/support.html
> > > > > Yahoo! Groups Links
> > > > > 
> > > > > 
> > > > > 
> > > > >
> > > >
> > >
> >
>



------------------------------------

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/