[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: Best Fitness Function



PureBytes Links

Trading Reference Links

Hi,

Did you end up posting that DLL (or its source) anywhere? I didn't 
notice it go by. I don't receive attachments as part of these posts, 
but I looked in the messaging archive at purebytes and did not see 
any attachment there either.

Thanks.

Mike

--- In amibroker@xxxxxxxxxxxxxxx, "dloyer123" <dloyer123@xxx> wrote:
>
> Thanks for the great response.
> 
> With so many options available, I like the idea of have a "best" 
> solution with some logical or at least empirical reasoning.  
> 
> I am doing some walkforward testing now with UPI, to see how it 
works.
> 
> So far, Car/Mdd and Pessimistic Car/Mdd seem to have good results 
in 
> the walk forward testing.  k-ratio was disappointing.  
> 
> Mdd does have a "long tail" on the positive side, like a bell curve 
> with one end pulled out.  The car distribution is much 
> more "normal".  
> 
> Great link to Howards's slides.  I enjoyed his book and look 
forward 
> to the enxt one.
> 
> I dont mind sharing the dll for Pesimistic Car/Mdd.  There are also 
> some performance hacks to speed up DayOfWeek(), RoundPenny() and 
> IsOptionsWeek() that togeher, shaved 20% of the execution time off 
my 
> system.  
> 
> --- In amibroker@xxxxxxxxxxxxxxx, "Mike" <sfclimbers@> wrote:
> >
> > Hi,
> > 
> > I think that the answer that you are most likely to hear is that 
> > there is no 'best' fitness function, since no two traders are 
> exactly 
> > alike, and the function has to suit the trader.
> > 
> > However, generalizations can be made with regards to 
> characteristics 
> > that some fitness functions appear to posess. For example; K-
Ratio 
> > and UPI tend to have a smooth steady slope.
> > 
> > ***
> > 
> > Currently, I have been striving to minimize the size, and more 
> > importantly to me - the duration, of a drawdown. As such, I find 
> that 
> > UPI serves as a good base. A nice description written by Peter 
> > Martin, the creator of the Ulcer Index and UPI (also known as 
> Martin 
> > Ratio), can be found here: http://www.tangotools.com/ui/ui.htm
> > 
> > I combine UPI with filters unique to my needs (with regards to 
> > holding periods and number of trades). For exmaple, I deduct from 
> the 
> > UPI on a sliding scale when there are too few trades or when 
trades 
> > are on average held too long. I produce a custom metric, using 
> custom 
> > backtester code, which I then use as the fitness function for 
walk 
> > forward.
> > 
> > ***
> > 
> > Howard Bandy regularly cites CAR/MDD, RAR/MDD, RRR, Ulcer 
> Performance 
> > Index, and K-Ratio as strong starting points from which to form 
> your 
> > personalized metric. He gave a nice example in his Las Vegas 
> workshop 
> > (slides for a similar workshop can be found in this post: 
> > http://finance.groups.yahoo.com/group/amibroker/message/123602 ) 
> > where he outlined a custom metric based on the following criteria:
> > 
> > "My goal is to make a 15% annual profit trading common stocks, 
> > control drawdowns, cherry-pick trades, hold about one week, and 
be 
> > tradable without interfering with my day job."
> > 
> > Howard also offered a suggestion for evaluating your fitness 
> function 
> > by running several different functions against the same data, 
> > printing out the graphs, and then eyeballing the equity curves to 
> see 
> > which one was most appealing to you. In practice you may be 
> surprised 
> > at what ultimately appeals to you.
> > 
> > ***
> > 
> > In his most recent book (The Evaluation and Optimization of 
Trading 
> > Strategies), Robert Pardo offers up a suggested metric that he 
> calls 
> > PROM (Pessimistic return on margin) which is very much like what 
> you 
> > are describing in that he adjusts downwards the number of winning 
> > trades (by the square root of the number of winners) before 
> > multiplying by the average win. Similarly, he adjusts upwards (by 
> the 
> > square root) the number of losing trades before multiplying by 
the 
> > average loss. Using this adjusted gross return, he then 
calculates 
> > the annualized rate of return over margin (assuming futures 
> trading).
> > 
> > Pardo also advocates measuring performance against perfect profit 
> > where perfect profit is defined as the sum total of all of the 
> > potential profit that could be realized by buying every bottom 
and 
> > selling every top. He refers to the ratio of Net Profit/Perfect 
> > Profit as "model efficiency".
> > 
> > ***
> > 
> > In direct contrast to many of the measures above, Ralph Vince 
(The 
> > Handbook of Portfolio Mathematics) belittles any calculation 
using 
> > MDD as being delusional. 50 tosses of a fair coin can and will 
> result 
> > in 50 straight losses from time to time. Does that imply that the 
> > probabilities have changed at all just because your backtesting 
> only 
> > saw 25 losing tosses earlier?
> > 
> > He states that two systems are best compared by determining their 
> > geometric means as calculated at their "optimal f", and the 
> > comparrison of the optimal f itself, on a two dimensional scale 
> such 
> > that the higher geometric mean with the lower optimal f is the 
> better 
> > system.
> > 
> > I haven't finished this book yet, but he introduces a revised 
> > approach later which addresses the very real problem of optimal f 
> in 
> > that the drawdowns can be severe, thereby precluding most traders 
> > from following through on the principle.
> > 
> > ***
> > 
> > Your DLL sounds appealing. Would you be willing to share it with 
> > others?
> > 
> > Mike 
> > 
> > --- In amibroker@xxxxxxxxxxxxxxx, "dloyer123" <dloyer123@> wrote:
> > >
> > > Has anyone done quantitative testing to pick the best fitness 
> > > function?
> > > 
> > > Car/mdd seems to work well, better than k-ratio or sharpe in my 
> > > personal testing.
> > > 
> > > Haven't tried Ulcer performance index.
> > > 
> > > I just woundered what others have found.
> > > 
> > > Since the fitness function has such a major impact on the 
system 
> > that 
> > > results, it seems that a good fitness function should help the 
> > system 
> > > perform well in walk forward testing.  This could be measured 
by 
> > walk 
> > > forward testing the same system with different fitness 
functions 
> > and 
> > > comparing how they do in the out of sample data.
> > > 
> > > I have been experimenting with a new fitness function I 
> > > call "pessimistic car/mdd".  It uses a dll to resample the 
trade 
> > list 
> > > after each run 10,000 times, similar to a bootstrap method.  It 
> > finds 
> > > the histogram of the resampled car and mdd values, then 
> calculates 
> > > car/mdd based on average car - 1 stdev and mdd + 1 stdev.  
> > > 
> > > The goal of this fitness function is to minimize the impact of 
> data 
> > > mining.
> > >
> >
>



------------------------------------

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/