--- In
amibroker@xxxxxxxxxxxxxxx, "Howard B" <howardbandy@xxx> wrote:
>
> Greetings all --
>
> I received the following email privately. I think the discussion is
> important enough to post it to the group.
>
> //--------------------------------------
>
> Dear Howard,
>
> Thanks for your reply in the Amibroker forum. The topic of how to
> sort optimisation results is an important one.
>
> You mention several statistics:
>
> KRatio, RRR, UPI, CAR/MDD, RAR/MDD, Recovery Ratio.
>
> Do you have any experience in using trading systems optimised to one
> of these?
>
> I'm finding that by optimising for highest account value, I go
> through long periods of drawdown, then a small % of trades make a
> killing (eg: 46% wins, 2:1 Win to Loss ratio overall). This is not
> what I'm looking for.
>
> I'd like to try and smooth out the equity curve to give more of a
> balance between making a large capital gain and also regular
> cashflow. I have the following stats:
>
> - % Wins
> - Win:Loss Ratio (All profits/All losses)
> - Risk:Reward
>
> I have found that by sorting results by Risk:Reward does not give
> good overall account balances.
>
> //-----------------------------------------------
>
> The experience that writer has is exactly the point I am making
when I say
> that net profit is usually a poor objective function to use when
developing
> trading systems.
>
> Each of the metrics I mentioned have three positive characteristics:
> 1. they reward equity growth
> 2. they penalize drawdowns
> 3. when used as The objective function for walk forward testing,
they tend
> to select systems that perform well out-of-sample
>
> My feeling is that the choice of objective comes very early on in
the
> design, test, and validation process. The objective function
incorporates
> the features that the trader wants in his or her trading, so
systems that
> rank well by using the score of that objective function are systems
that the
> person knows they will be comfortable with.
>
> If one of those metrics already built in to AmiBroker is not
satisfactory,
> it is easy to create your own metric and have AmiBroker report it
for every
> run and use it in the walk forward testing. The custom metric can
include
> setting limits on percent winners, win to loss ratio, and so forth.
>
> There are several posts over the past few days that discuss this
and show
> examples of how to do it.
>
> Try out whatever you think might work for you. Keep in mind that
optimizing
> using an objective function that does not include some penalty for
drawdowns
> and / or reward for equity smoothness is likely to result in
systems that
> have very strange results and do not perform well out-of-sample.
>
> As always, be sure to evaluate the results by examining out-of-
sample
> results -- in-sample results have no value in estimating the likely
> out-of-sample performance. NO value. Trading based on in-sample
results
> alone is a certain way to lose your trading account. (Running
Monte Carlo
> permutations on in-sample results does Nothing to improve the
likelihood of
> out-of-sample profitability.)
>
> Thanks for listening,
> Howard
>
www.quantitativetradingsystems.com
>
>
>