|
PureBytes Links
Trading Reference Links
|
Hi,
I think that the answer that you are most likely to hear is that
there is no 'best' fitness function, since no two traders are exactly
alike, and the function has to suit the trader.
However, generalizations can be made with regards to characteristics
that some fitness functions appear to posess. For example; K-Ratio
and UPI tend to have a smooth steady slope.
***
Currently, I have been striving to minimize the size, and more
importantly to me - the duration, of a drawdown. As such, I find that
UPI serves as a good base. A nice description written by Peter
Martin, the creator of the Ulcer Index and UPI (also known as Martin
Ratio), can be found here: http://www.tangotools.com/ui/ui.htm
I combine UPI with filters unique to my needs (with regards to
holding periods and number of trades). For exmaple, I deduct from the
UPI on a sliding scale when there are too few trades or when trades
are on average held too long. I produce a custom metric, using custom
backtester code, which I then use as the fitness function for walk
forward.
***
Howard Bandy regularly cites CAR/MDD, RAR/MDD, RRR, Ulcer Performance
Index, and K-Ratio as strong starting points from which to form your
personalized metric. He gave a nice example in his Las Vegas workshop
(slides for a similar workshop can be found in this post:
http://finance.groups.yahoo.com/group/amibroker/message/123602 )
where he outlined a custom metric based on the following criteria:
"My goal is to make a 15% annual profit trading common stocks,
control drawdowns, cherry-pick trades, hold about one week, and be
tradable without interfering with my day job."
Howard also offered a suggestion for evaluating your fitness function
by running several different functions against the same data,
printing out the graphs, and then eyeballing the equity curves to see
which one was most appealing to you. In practice you may be surprised
at what ultimately appeals to you.
***
In his most recent book (The Evaluation and Optimization of Trading
Strategies), Robert Pardo offers up a suggested metric that he calls
PROM (Pessimistic return on margin) which is very much like what you
are describing in that he adjusts downwards the number of winning
trades (by the square root of the number of winners) before
multiplying by the average win. Similarly, he adjusts upwards (by the
square root) the number of losing trades before multiplying by the
average loss. Using this adjusted gross return, he then calculates
the annualized rate of return over margin (assuming futures trading).
Pardo also advocates measuring performance against perfect profit
where perfect profit is defined as the sum total of all of the
potential profit that could be realized by buying every bottom and
selling every top. He refers to the ratio of Net Profit/Perfect
Profit as "model efficiency".
***
In direct contrast to many of the measures above, Ralph Vince (The
Handbook of Portfolio Mathematics) belittles any calculation using
MDD as being delusional. 50 tosses of a fair coin can and will result
in 50 straight losses from time to time. Does that imply that the
probabilities have changed at all just because your backtesting only
saw 25 losing tosses earlier?
He states that two systems are best compared by determining their
geometric means as calculated at their "optimal f", and the
comparrison of the optimal f itself, on a two dimensional scale such
that the higher geometric mean with the lower optimal f is the better
system.
I haven't finished this book yet, but he introduces a revised
approach later which addresses the very real problem of optimal f in
that the drawdowns can be severe, thereby precluding most traders
from following through on the principle.
***
Your DLL sounds appealing. Would you be willing to share it with
others?
Mike
--- In amibroker@xxxxxxxxxxxxxxx, "dloyer123" <dloyer123@xxx> wrote:
>
> Has anyone done quantitative testing to pick the best fitness
> function?
>
> Car/mdd seems to work well, better than k-ratio or sharpe in my
> personal testing.
>
> Haven't tried Ulcer performance index.
>
> I just woundered what others have found.
>
> Since the fitness function has such a major impact on the system
that
> results, it seems that a good fitness function should help the
system
> perform well in walk forward testing. This could be measured by
walk
> forward testing the same system with different fitness functions
and
> comparing how they do in the out of sample data.
>
> I have been experimenting with a new fitness function I
> call "pessimistic car/mdd". It uses a dll to resample the trade
list
> after each run 10,000 times, similar to a bootstrap method. It
finds
> the histogram of the resampled car and mdd values, then calculates
> car/mdd based on average car - 1 stdev and mdd + 1 stdev.
>
> The goal of this fitness function is to minimize the impact of data
> mining.
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|