|
PureBytes Links
Trading Reference Links
|
Pal:
The success or
failure of many different mechanical systems is predicated to a surprising
and varying degree on the sequence of events just prior to the first actual
trade generated by the system. The trade setup and timing of the first
trade can have a profound effect on the subsequent trading results. The
circumstances and timing of entry into the first trade can sometimes make a
huge difference in the overall trading performance.
You've spotlighted a simple truth: depending
on what day you start your test, results will differ. Any profound effects
on resullts should be a red flag. If the starting date of your data set
adversely affects your returns, I would assume the system is not robust.
"Robust" is tough term to define...but, yes, results vary depending on starting
dates. Drastic differences in returns should make one very suspect of
their system design.
Take care,
Steve
<BLOCKQUOTE
>
----- Original Message -----
<DIV
>From:
palsanand
To: <A title=amibroker@xxxxxxxxxxxxxxx
href="">amibroker@xxxxxxxxxxxxxxx
Sent: Sunday, October 19, 2003 11:44
AM
Subject: Objective functions (was RE:
[amibroker] Re: Optimization -- again)
Hi,Here is an interesting observation on system
testing:Say you run a system test over 10,000 bars of data, then print
out a chart of the system's equity line. Then repeat the test, but start
100 bars later. Let's say two trades were included in those 100 bars,
so they've been dropped. Now print the second equity line and compare
it to the first. You'd get exactly the same equity line, but 100 bars
shorter. Right? Wrong! When I do this I get a radically different
equity line on the second test, i.e., they are not near-mirror images of
each other. My hunch is that a form of the chaotician's "butterfly-effect"
has arisen: changing any given trade's market position (long, short, flat)
will effect in a chain reaction all the subsequent trades in complex and
unexpected ways. Here dropping the first two trades could very well
change the system's market position when the third trade is
calculated, and so on. I believe this observation has profound and
unfortunate implications for the robustness of system testing. It's a
second and more subtle problem that lies behind the mere
curve-fitting/optimization problem. If dropping a couple of early
trades will always effect later trades, then there's no truly "neutral"
starting point with any test data. Where your test data starts determines
the final test results just as much as your system does. The
success or failure of many different mechanical systems is predicated to a
surprising and varying degree on the sequence of events just prior to the
first actual trade generated by the system. The trade setup and timing
of the first trade can have a profound effect on the subsequent trading
results. The circumstances and timing of entry into the first trade can
sometimes make a huge difference in the overall trading performance.
Regards,Pal--- In amibroker@xxxxxxxxxxxxxxx, "Fred"
<fctonetti@xxxx> wrote:> That IS what I was trying to say.
I suspect because equity feed back > is like looking in a rear view
mirror, great for letting us know > where we were and how we
could have adjusted the past to make it > better, but that's about
it. > > --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill"
<dmerrill@xxxx> > wrote:> > don't think I get what you
mean here fred.> > > > you can't be saying that metrics on
the equity curve of a trading > strategy> > or its parameters
aren't useful, right? that's the only thing we > have to> >
judge the effectiveness of our methods and settings.> > >
> so you must be saying that equity feedback isn't a useful
concept,> > regardless of how you measure "good" equity. do I have
that right?> > > > if so, as I've said, my experience
agrees -- none of the indicators > I've> > tried are
wonderfully profitable when auto-optimized this way. I > just
cannot> > for the life of me understand why that's the case, if
backtests > tell us> > anything useful about future
performance.> > > > if I've misunderstood completely, my
apoligies (:-)> > > > dave> > Like a
lot of other things that sound like they SHOULD work, I > have>
> never found metrics related to equity curve feedback to be of
much> > value in the determination of system
parameter values.> > > > --- In
amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx>>
> wrote:> > > interesting as usual
howard (:-). one piece I wanted to drill > into>
> a bit.> > >> >
> I wonder what the effect of using performance measures that>
> concentrate on> > > certain things
at the expense of others actually is.> > >>
> > for example, my auto-optimization stuff currently uses
simple> > profit per bar> > >
to choose parameter values. my gut-level assumption was that >
since> > it was> > > ignoring
drawdown (among other things), the resulting systems > might>
> have> > > higher drawdown than I was
comfortable with, but that profit per> > bar
should> > > be as good as the trading method could
produce.> > >> > > maybe
that's not the case. maybe by choosing a more balanced>
> success metric,> > > not only would
the other factors not considered by my simplistic> >
first pass> > > metric be improved, but profitability
might be improved as well.> > >>
> > is this something you've investigated or thought about?
anyone > else?> > >>
> > dave> > > Note ? it
is perfectly valid to have different objective> >
functions for> > > different purposes. For
example, I might be modeling the > behavior> > of
a> > > sector, say oil services, with the intent of
trading individual> > stocks based>
> > on what I learn. In this case, I want to identify
periods of> > rising prices> >
> with careful attention to turning points, but without much >
interest> > in> > > overall
profit. On the other hand, I might be modeling >
individual> > high beta> > >
tech stocks, in which case my model includes several stop loss>
> techniques> > > and I care most
about avoiding drawdowns.> > >>
> >> > >> >
> Thanks,> > >>
> > Howard> > > > >
> Yahoo! Groups
Sponsor> > > > > > > >
Send BUG REPORTS to bugs@xxxx> > Send SUGGESTIONS to
suggest@xxxx> >
-----------------------------------------> > Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx>
> (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)>
> -------------------------------------------->
> Check group FAQ at:> > <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html>
> > > Your use of Yahoo! Groups is subject to the
Yahoo! Terms of > Service.Send
BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Yahoo! Groups Sponsor
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
|