|
PureBytes Links
Trading Reference Links
|
Yes, In fact, I am of the opinion that IF one strongly believed in
optimization, then one should use maximum period available. I did say
that AO is probably an overkill, but I can see your logic in the
question "why not"?
> to me, it keeps boiling down to the same thing: if past performance
predicts
> future performance, AO should work. if AO doesn't work, why doesn't
that say
> that past performance doesn't predict future behavior?
Is your fundamental question -- why doesn't it work in ALL CASES,
regardless of the underlying strategy? If yes, then the answer is
clearly no, it does not. There could be only very few number of
truely recurring patterns with certain parameters in the market
you're testing. If every pattern that "seems to" recur could be
traded profitably by optimizing, then everyone with access to an
Optimize button would be a successful trader.
I guess what I'm trying to say is that your technique could be very
very profitable with certain strategies that you have yet to discover
(and I'm sure you will). Its the mind blowing possibilities to try is
what frustrates all of us. And I know you must have tried a whole lot
of them so far, but I wouldn't give up.
Jitu
--- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx>
wrote:
> if optimization makes sense at all, why does it make sense to
optimize on
> one particular day, but not another? which day would you choose? if
it's
> useful the first day, why then not on the second?
>
> note that if you always optimize on all data before the current day,
> typically after a while things stabilize and don't change much. any
new
> behavior has to persist for quite a while for it to overcome the
weight of
> all that prior history.
>
> re forward testing, auto-optimization (I'll abbreviate AO) makes
the entire
> test a forward test, in the sense that every day, you're trading
out of
> sample data. you never optimize on future data, so your
optimizations
> reflect only prior history.
>
>
> if you think about the whole process of optimization and testing,
it goes
> like this: optimize up to some date, trade those settings for some
period of
> time after that, optimize up to another later date, use those
settings for
> some period of time, etc. how do you pick the times to optimize vs
times to
> test? if it's a valid approach, can't you pick any optimization
period and
> have the same odds of success?
>
> now suppose you wanted to know whether the entire approach of
optimization
> worked at all. so you test that, by choosing a series of
optimization
> periods and examining performance with those settings after that.
AO just
> automates that whole process, while remaining completely agnostic
about
> which optimization period you choose, by choosing them all, one day
at a
> time. in a pretty real sense, AO is a backtest of the idea of
optimization.
> if AO fails to produce good results, why isn't that an indictment of
> optimization itself?
>
>
> to me, it keeps boiling down to the same thing: if past performance
predicts
> future performance, AO should work. if AO doesn't work, why doesn't
that say
> that past performance doesn't predict future behavior?
>
> this is the reason I glommed optimization and backtesting together
in some
> senses. what's the use of either backtesting or optimization if the
past
> doesn't tell us much about the future?
>
> I'm stuck in a mental loop. I keep saying the same things over and
over, as
> I'm sure you've noticed. either I believe the conclusions this line
of
> thinking led me to, or I don't. if I do, logic says I abandon TA
completely
> and look only to fundamentals, industry understanding, and insider
behavior
> to make investment decisions. if I think there's a hole in the logic
> instead, I have to find it, or I can't distinguish meaningful tests
from
> whatever mistake led me to the conclusion I don't accept.
>
>
> re your last question about choosing between optimization results,
the case
> where you optimize over all 3 years, including the test year, is
invalid, or
> at least it's not a forward test. as we keep coming back to, of
course you
> do better when you optimize over the period you're going to test,
because
> your settings accommodate what happened during that time. only
tests on data
> that wasn't used to generate the settings you're using are valid,
IMO.
>
> dave
> Aren't you mixing the two -- backtest and optimization? The
backtest
> is of course, absolutely necessary. Probably same goes for a
little
> bit of optimization. But isn't auto-optimization every day the
other
> end of the spectrum? While backtest in itself is a form of
> optimization (with just one set of parameters), auto-optimization
> every day comes pretty close to serious curve-fitting, IMHO.
>
> In one of your other posts, I think you said that you got the best
> results when you optimized over entire dataset. Assuming you meant
> that when you used last X number of days/months as a forward-test
OOS
> period, I wasn't too surprised to hear that. Why wouldn't you use
all
> the past data to find optimum parameters if you're going to try to
> auto-optimize? Of course, you could then try to find the optimum
> period to optimize over, but that too again might be curve-
fitting.
>
> I have a question for proponents of back and forward testing.
Assume
> that we're dealing with 3 years of data, where we use first two
years
> to optimize and the third year as forward testing period. Further
> assume that you get following results -
>
> 1. When optimized over first two years, both of those years show
> profit, but the third year shows loss.
> 2. When optimized over all three years, all three years show
profit,
> although less than that showed by first test in first two years.
>
> Which set of parameters would you use going forward in the fourth
> year? Or would you rather just dump the strategy? I personally
think
> that back and forward testing makes sense ONLY if you do the
latter,
> but I'm not sure that's what happens. But then... I could be
wrong.
>
> Jitu
>
> --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx>
> wrote:
> > I'm willing to do the work I think, have been trying to
already, to
> the best
> > of my limited understanding.
> >
> > but it's the concept of what to look for and how that puzzles
me.
> sometimes
> > I do get nice backtest numbers, at least moderately nice. but
if I
> don't
> > understand why that should be possible given what's been said
about
> the
> > dynamics of market evolution, why should I believe them?
> >
> > I keep coming back to the same question that I'm sure you're all
> sick to
> > death of, I know I am: if auto-optimization isn't very
successful,
> doesn't
> > that imply that past performance tells us very little about the
> future, and
> > if that's the case, how do we develop trading systems?
> >
> > dave
> >
> > > ...frankly, it's hard to see how rational
> > > trading system design is possible in
> > > a world like this. or am I just depressed?
> >
> > I like Edison's attitude when looking for the right material
to
> use
> > in the light bull. When yet another "bright" idea (sorry I
could
> not
> > resist) failed when tested, he is reported to have said, "We
are
> > making progress. We now know of 999 things that will not
work."
> > After trying everything from bamboo to who knows what, he
> eventually
> > found the right material.
> >
> > Genius is 5% inspiration and 95% perspiration.
> >
> > b
>
>
> Yahoo! Groups Sponsor
> ADVERTISEMENT
>
>
>
>
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
>
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of
Service.
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|