|
PureBytes Links
Trading Reference Links
|
----- Original Message -----
From: "jtelang" <<A
href=""><FONT
size=2>jtelang@xxxxxxxxx<FONT
size=2>>
To: <<A
href=""><FONT
size=2>amibroker@xxxxxxxxxxxxxxx<FONT
size=2>>
Sent: Friday, October 17, 2003 6:17
PM
Subject: [amibroker] Re: Will systems degrade? (was
Optimization)
> I'm not sure why it matters. In this theoretical
scenario, assume all > 3 years are of same type, whether bull, bear, or
trendless (or better > yet, combination of all 3) and you are assured
that the fourth year > also has same behavior. I'm sure you'll agree that
even if the market > behavior is same, one could be easily trying a
strategy that displays > the behavior I described (it may not be a good
system, but that's not > known prior to testing it anyway).
> > The question is more
along the lines of "should one consider the new > set of parameters that
also took third year's data into account" > OR "should one strictly stick
to the principle of forward testing and > dump the system (because it
didn't work in the OOS period of 3rd > year)"?
IMO, it matters because up, down,
sideways markets can have remarkably different characteristics
that interact with the system's rules.
<FONT
color=#800000 size=2>If the years are the same, then I would inspect the
rules and find out why they produced different results in the last period.
If this is successful, modify the system and try again. If not, dump
the system and move on.
> > Jitu> > --- In <A
href=""><FONT
size=2>amibroker@xxxxxxxxxxxxxxx,
"wavemechanic" <<FONT
size=2>wd78@x...> wrote:>
> > > ----- Original Message ----- > > From: "jtelang"
<<FONT
size=2>jtelang@x...>> >
To: <<FONT
size=2>amibroker@xxxxxxxxxxxxxxx<FONT
size=2>>> > Sent: Friday, October 17, 2003 4:53 PM> >
Subject: [amibroker] Re: Will systems degrade? (was Optimization)> >
> > > > snip> > > > > I have a
question for proponents of back and forward testing. > Assume>
> > that we're dealing with 3 years of data, where we use first two
> years> > > to optimize and the third year as forward
testing period. Further> > > assume that you get following results
-> > >> > > 1. When optimized over first two years,
both of those years show> > > profit, but the third year shows
loss.> > > 2. When optimized over all three years, all three years
show > profit,> > > although less than that showed by first
test in first two years.> > >> > > Which set of
parameters would you use going forward in the fourth> > > year? Or
would you rather just dump the strategy? I personally > think>
> > that back and forward testing makes sense ONLY if you do the >
latter,> > > but I'm not sure that's what happens. But then... I
could be > wrong.> > > > The first question is does
the third year differ from the first > two. For> >
example, do the first two years trend up and the third down. If >
things are> > going to work OK, the two periods should be
similar. For this > reason, some> > use bull markets to
develop bull market systems, etc., and others > use a> >
sufficiently large data set that encompasses bull, bear, and >
trendless> > markets to develop systems. The first approach
would appear to be > more> > aggressive, if one can always be
sure what type of market one is > in. The> > other has a
shot at producing a robust system that rides through > all types>
> of markets. Choose your poison since there is no absolute
answer.> > > > >> > > Jitu> >
>> > > --- In <A
href=""><FONT
size=2>amibroker@xxxxxxxxxxxxxxx, "Dave
Merrill" <<FONT
size=2>dmerrill@x...>> >
> wrote:> > > > I'm willing to do the work I think, have been
trying to > already, to> > > the best> > > >
of my limited understanding.> > > >> > > > but
it's the concept of what to look for and how that puzzles > me.>
> > sometimes> > > > I do get nice backtest numbers, at
least moderately nice. but > if I> > > don't> >
> > understand why that should be possible given what's been said >
about> > > the> > > > dynamics of market evolution,
why should I believe them?> > > >> > > > I keep
coming back to the same question that I'm sure you're all> > > sick
to> > > > death of, I know I am: if auto-optimization isn't very
> successful,> > > doesn't> > > > that imply
that past performance tells us very little about the> > > future,
and> > > > if that's the case, how do we develop trading
systems?> > > >> > > > dave> > >
>> > > > > ...frankly, it's hard to see how
rational> > > > > trading system design is
possible in> > > > > a world like this. or am I
just depressed?> > > >> > > > I like
Edison's attitude when looking for the right material > to> >
> use> > > > in the light bull. When yet another
"bright" idea (sorry I > could> > > not> > >
> resist) failed when tested, he is reported to have said, "We
> are> > > > making progress. We now know of
999 things that will not > work."> > > >
After trying everything from bamboo to who knows what, he> > >
eventually> > > > found the right material.>
> > >> > > > Genius is 5% inspiration and
95% perspiration.> > > >> > > >
b> > >> > >> > >> > > Send
BUG REPORTS to <FONT
size=2>bugs@x...> > > Send
SUGGESTIONS to <FONT
size=2>suggest@x...> > >
-----------------------------------------> > > Post
AmiQuote-related messages ONLY to: <A
href=""><FONT
size=2>amiquote@xxxxxxxxxxxxxxx>
> > (Web page: <A
href=""><FONT
size=2>http://groups.yahoo.com/group/amiquote/messages/<FONT
size=2>)> > > -------------------------------------------->
> > Check group FAQ at:> > <A
href=""><FONT
size=2>http://groups.yahoo.com/group/amibroker/files/groupfaq.html<FONT
size=2>> > >> > > Your use of Yahoo! Groups is subject to
> <A
href=""><FONT
size=2>http://docs.yahoo.com/info/terms/<FONT
size=2>> > >> > >> > >
------------------------ Yahoo! Groups Sponsor
---------------------~-->> Rent DVDs from home.> Over 14,500
titles. Free Shipping> & No Late Fees. Try Netflix for FREE!>
<A
href=""><FONT
size=2>http://us.click.yahoo.com/mk9osC/hP.FAA/3jkFAA/GHeqlB/TM<FONT
size=2>>
---------------------------------------------------------------------~->>
> Send BUG REPORTS to <A
href=""><FONT
size=2>bugs@xxxxxxxxxxxxx> Send
SUGGESTIONS to <A
href=""><FONT
size=2>suggest@xxxxxxxxxxxxx>
-----------------------------------------> Post AmiQuote-related messages
ONLY to: <FONT
size=2>amiquote@xxxxxxxxxxxxxxx
> (Web page: <A
href=""><FONT
size=2>http://groups.yahoo.com/group/amiquote/messages/<FONT
size=2>)> --------------------------------------------> Check
group FAQ at: <A
href=""><FONT
size=2>http://groups.yahoo.com/group/amibroker/files/groupfaq.html<FONT
size=2> > > Your use of Yahoo! Groups is subject to
<FONT
size=2>http://docs.yahoo.com/info/terms/<FONT
size=2> > >
Yahoo! Groups Sponsor
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
|