|
PureBytes Links
Trading Reference Links
|
Dave,
Aren't you mixing the two -- backtest and optimization? The backtest
is of course, absolutely necessary. Probably same goes for a little
bit of optimization. But isn't auto-optimization every day the other
end of the spectrum? While backtest in itself is a form of
optimization (with just one set of parameters), auto-optimization
every day comes pretty close to serious curve-fitting, IMHO.
In one of your other posts, I think you said that you got the best
results when you optimized over entire dataset. Assuming you meant
that when you used last X number of days/months as a forward-test OOS
period, I wasn't too surprised to hear that. Why wouldn't you use all
the past data to find optimum parameters if you're going to try to
auto-optimize? Of course, you could then try to find the optimum
period to optimize over, but that too again might be curve-fitting.
I have a question for proponents of back and forward testing. Assume
that we're dealing with 3 years of data, where we use first two years
to optimize and the third year as forward testing period. Further
assume that you get following results -
1. When optimized over first two years, both of those years show
profit, but the third year shows loss.
2. When optimized over all three years, all three years show profit,
although less than that showed by first test in first two years.
Which set of parameters would you use going forward in the fourth
year? Or would you rather just dump the strategy? I personally think
that back and forward testing makes sense ONLY if you do the latter,
but I'm not sure that's what happens. But then... I could be wrong.
Jitu
--- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx>
wrote:
> I'm willing to do the work I think, have been trying to already, to
the best
> of my limited understanding.
>
> but it's the concept of what to look for and how that puzzles me.
sometimes
> I do get nice backtest numbers, at least moderately nice. but if I
don't
> understand why that should be possible given what's been said about
the
> dynamics of market evolution, why should I believe them?
>
> I keep coming back to the same question that I'm sure you're all
sick to
> death of, I know I am: if auto-optimization isn't very successful,
doesn't
> that imply that past performance tells us very little about the
future, and
> if that's the case, how do we develop trading systems?
>
> dave
>
> > ...frankly, it's hard to see how rational
> > trading system design is possible in
> > a world like this. or am I just depressed?
>
> I like Edison's attitude when looking for the right material to
use
> in the light bull. When yet another "bright" idea (sorry I could
not
> resist) failed when tested, he is reported to have said, "We are
> making progress. We now know of 999 things that will not work."
> After trying everything from bamboo to who knows what, he
eventually
> found the right material.
>
> Genius is 5% inspiration and 95% perspiration.
>
> b
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|