|
PureBytes Links
Trading Reference Links
|
Pal,
I certainly do not see (and understand why) the results would be
RADICALLY different. I tried what you suggested, and obviously, near
the start of second period, the results were slightly different, but
eventually, the system caught up, and produced nearly identical
equity curve as the first test.
Jitu
--- In amibroker@xxxxxxxxxxxxxxx, "palsanand" <palsanand@xxxx> wrote:
> Hi,
>
> Here is an interesting observation on system testing:
>
> Say you run a system test over 10,000 bars of data, then print out
a
> chart of the system's equity line. Then repeat the test, but start
> 100 bars later. Let's say two trades were included in those 100
bars,
> so they've been dropped. Now print the second equity line and
compare
> it to the first. You'd get exactly the same equity line, but 100
bars
> shorter. Right? Wrong!
>
> When I do this I get a radically different equity line on the
second
> test, i.e., they are not near-mirror images of each other. My hunch
> is that a form of the chaotician's "butterfly-effect" has arisen:
> changing any given trade's market position (long, short, flat) will
> effect in a chain reaction all the subsequent trades in complex and
> unexpected ways. Here dropping the first two trades could very well
> change the system's market position when the third trade is
> calculated, and so on.
>
> I believe this observation has profound and unfortunate
implications
> for the robustness of system testing. It's a second and more subtle
> problem that lies behind the mere curve-fitting/optimization
problem.
>
> If dropping a couple of early trades will always effect later
trades,
> then there's no truly "neutral" starting point with any test data.
> Where your test data starts determines the final test results just
as
> much as your system does.
>
> The success or failure of many different mechanical systems is
> predicated to a surprising and varying degree on the sequence of
> events just prior to the first actual trade generated by the
system.
>
> The trade setup and timing of the first trade can have a profound
> effect on the subsequent trading results. The circumstances and
> timing of entry into the first trade can sometimes make a huge
> difference in the overall trading performance.
>
> Regards,
>
> Pal
>
> --- In amibroker@xxxxxxxxxxxxxxx, "Fred" <fctonetti@xxxx> wrote:
> > That IS what I was trying to say. I suspect because equity feed
> back
> > is like looking in a rear view mirror, great for letting us know
> > where we were and how we could have adjusted the past to make it
> > better, but that's about it.
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx>
> > wrote:
> > > don't think I get what you mean here fred.
> > >
> > > you can't be saying that metrics on the equity curve of a
trading
> > strategy
> > > or its parameters aren't useful, right? that's the only thing
we
> > have to
> > > judge the effectiveness of our methods and settings.
> > >
> > > so you must be saying that equity feedback isn't a useful
concept,
> > > regardless of how you measure "good" equity. do I have that
right?
> > >
> > > if so, as I've said, my experience agrees -- none of the
> indicators
> > I've
> > > tried are wonderfully profitable when auto-optimized this way.
I
> > just cannot
> > > for the life of me understand why that's the case, if backtests
> > tell us
> > > anything useful about future performance.
> > >
> > > if I've misunderstood completely, my apoligies (:-)
> > >
> > > dave
> > > Like a lot of other things that sound like they SHOULD work,
I
> > have
> > > never found metrics related to equity curve feedback to be of
> much
> > > value in the determination of system parameter values.
> > >
> > > --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill"
<dmerrill@xxxx>
> > > wrote:
> > > > interesting as usual howard (:-). one piece I wanted to
drill
> > into
> > > a bit.
> > > >
> > > > I wonder what the effect of using performance measures that
> > > concentrate on
> > > > certain things at the expense of others actually is.
> > > >
> > > > for example, my auto-optimization stuff currently uses
simple
> > > profit per bar
> > > > to choose parameter values. my gut-level assumption was
that
> > since
> > > it was
> > > > ignoring drawdown (among other things), the resulting
systems
> > might
> > > have
> > > > higher drawdown than I was comfortable with, but that
profit
> per
> > > bar should
> > > > be as good as the trading method could produce.
> > > >
> > > > maybe that's not the case. maybe by choosing a more balanced
> > > success metric,
> > > > not only would the other factors not considered by my
> simplistic
> > > first pass
> > > > metric be improved, but profitability might be improved as
> well.
> > > >
> > > > is this something you've investigated or thought about?
> anyone
> > else?
> > > >
> > > > dave
> > > > Note ? it is perfectly valid to have different objective
> > > functions for
> > > > different purposes. For example, I might be modeling the
> > behavior
> > > of a
> > > > sector, say oil services, with the intent of trading
> individual
> > > stocks based
> > > > on what I learn. In this case, I want to identify periods
of
> > > rising prices
> > > > with careful attention to turning points, but without much
> > interest
> > > in
> > > > overall profit. On the other hand, I might be modeling
> > individual
> > > high beta
> > > > tech stocks, in which case my model includes several stop
loss
> > > techniques
> > > > and I care most about avoiding drawdowns.
> > > >
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Howard
> > >
> > >
> > > Yahoo! Groups Sponsor
> > >
> > >
> > >
> > > Send BUG REPORTS to bugs@xxxx
> > > Send SUGGESTIONS to suggest@xxxx
> > > -----------------------------------------
> > > Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx
> > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > --------------------------------------------
> > > Check group FAQ at:
> > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > >
> > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> > Service.
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Rent DVDs Online - Over 14,500 titles.
No Late Fees & Free Shipping.
Try Netflix for FREE!
http://us.click.yahoo.com/Tq9otC/XP.FAA/3jkFAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|