|
PureBytes Links
Trading Reference Links
|
Hi Brian,
Thank you for your comprehensive reply,
I agree with you that System Development and Evaluation (SD&E) and Money
Management (MM) are probably the most important aspects of freelance
trading. If you get these right most of the rest is operational.
At this time I thought my question was really about SD&E rather than MM,
although after reading your post I realise that I have, and will always
have, MM issues incorporated in the model under test.
I see your point of using core metrics, however if I can I would like to
avoid your step of going out to Excel to manage the MM part.
I have used back-testing over a the last eight years firstly with
Metastock/Tradesim then Wealth-lab Developer and now AmiBroker. In starting
to use AmiBroker I am perhaps having a harder look at the processes.
Do you see any value, for example, in constructing a CBT added metric for
exposure based on "clamped" equity? My thought was that if I did this,
along with using the "clamped" model virtually all measure would remain
valid over the life of the model even in Walk-forward testing. Can you see
any problems in this approach?
Regards,
Robert
Melbourne, Au
From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf
Of brian_z111
Sent: Friday, 9 May 2008 11:37
To: amibroker@xxxxxxxxxxxxxxx
Subject: [amibroker] Re: AB Back-testing Metrics
Hello Robert,
No, this is an excellent question.
The answer doens't come quickly though.
I will do what I can in a short time.
I specialise in system design/evaluation (SD&E) and MoneyManagement
(MM) because IMO they are the most important part of freelance
trading.
I find that they have not been treated very will in the general
trading literature (with some exceptions) - oversimplified, emphasis
on the wrong points or just plain wrong.
IMO this forum is doing a good job of SD&E with some people leading
the charge and Herman and Fred specialists in optimizing and high
freqency trading (which are closely related topics).
I agree that MM is a subject that is underdone.
So here is a starting point for your discussion on MM:
- A trading strategy produces a trade series, in backtesting, which
it the profile of that strategy.
- stops etc help shape that profile
- so the objective function/fitness measures (OF/FM) WE select will
directly impact on the profile (we can, but don't have to include
equity metrics in the OF/FM - that is a personal choice).
- once we have selected a model when intend to trade then the SD&E
phase is finished (for simplicity I will assume we are not going to
continue with ongoing optimisation, for now).
- my interest is in root cause evaluation and my argument is that the
profile can most easily and usefully be described using some core
metrics
- the core metrics are binomial factors (because trading is analgous
to a binomial event i.e. a coin flip
- core metrics are W/L ratio, PayOff ratio, ave time in trade and ave
time out of trade
Note: these can be expressed in different ways - if we use $ values
or % values, for example, we are assuming we are trading a fixed
equity account or a compounding equity account
- a metric like ProfitFactor is actually a standardised binomial
average of all trades in your backtest (I don't actually favour PF
entirely but it is a well known example).
Now the important part:
- the equity curve outcomes vary according to the MM rules we apply
(as you noted)
So, the profile is fixed (give or take variance) and is locked in at
the SD&E phase
The equity outcomes are dependent on the MM methods we use.
Refer to RalphVince for the classic example of how underbetting will
produce a smooth eq curve that doesn't go anywhere fast and how
overbetting will send you broke (even if the coin is heavily biased
in your favour) - I like his second book "The maths of MM etc"
At the moment I am exporting the trade series and doing MM in excel.
Later I will decide if I can do what I want to do within AB or not.
If not,it is possible I will work on a MM plugin (years away?).
There is too much going on for me to do it in AB at this stage but
others, who have been down that path might be able to help there.
My method:
- export the trade series, of a validated system I intend to trade,
as % Profit or Loss per trade
- do MM in excel
- the advantage is that without MM considerations within AB you can
return all trades (regardless of whether you had the eq to take them
or not) so you reach a statistically valid sample from a smaller
dataset (which is a good thing).
My proposition is that everything we can know, that is worth knowing,
can be gleaned from the core metrics.
This includes the trading edge we have (RV does a great job of
showing us how to convert any trading edge into a winning business -
of course the bigger the edge the better, relative to risk)
I hope that helps somewhat.
brian_z
--- In amibroker@xxxxxxxxxxxxxxx, "Robert Grigg" <robert@xxx> wrote:
>
> I have been thinking through the process of evaluating
the "goodness" of a
> trading system using AB metrics and have become perplexed. Can
someone who
> has unravelled this issue previously help?
>
> There seem to be two general approaches to portfolio sizing while
doing a
> back-test.
>
> The first is to only back-test using the "Initial Equity" amount.
> Generally, we might start using fixed position sizes and a fixed
maximum
> number of positions. In later developmental iterations we might use
risk
> based position sizing or other processes where we vary position
sizing up to
> the maximum amount of Initial Equity. I generally refer to this
evaluation
> approach as "Clamped Equity". This approach tends to give an
equity curve
> that is linear.
>
> The second approach is to compound profits and place trades up
to "Current
> Equity". (In AB terms our Position size is set to a % of Current
Equity).
> This is referred to as "Compounding Profits". The equity curve can
take on
> an exponential appearance.
>
> In real life trading most people tend to do a bit of both. However
in
> back-testing mode the "Compounding Profits" model (with a
notionally good
> system) can quickly become infeasible. (If only I had this system
in
> 2000...).
>
> So, now to the crux of the problem. The "Clamped Equity" approach,
with a
> notionally good system, produces a profit that is quarantined.
Accumulated
> profit can be used to top-up draw-downs but the amount in trades
never
> exceeds initial equity. In AmiBroker metrics, Exposure % is always
> calculated on a bar by bar basis of mark-to-market holding against
current
> mark-to-market equity. However, in the "Clamped Equity" testing
approach,
> the quarantining of profits is intentional and it seems to me that
it would
> be more useful to look at the Exposure as a % of the "Clamped
Equity" (i.e.
> the "Initial Equity")?
>
> Exposure% is also used as a divisor in other metrics such as Net
Risk
> Adjusted Return %, Risk Adjusted Return %, Max System % Draw-down,
> CAR/MaxDD and RAR/MaxDD and so these metrics also may be less
useful given
> this testing approach.
>
> I can see that comparisons between competing models, with the same
test
> period is valid. However, I do not feel so secure if I am doing
> Walk-Forward back-testing using a complex objective function,
particularly
> if I am using weighted components that contain Exposure% and others
that
> don't.
>
> I know that it is relatively easy to use the Custom Back Tester to
produce
> amended statistics. However, I am concerned that I have not found
any other
> discussions of this issue on this or other forums, so maybe I have
muddled
> thinking and it is not a real issue. Any discussion would be
appreciated.
>
> Robert
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|