|
PureBytes Links
Trading Reference Links
|
--- "Dave Merrill" <dmerrill@xxxx> wrote:
> I've thought about all this a great deal,
> and I really do wish somebody would help
> me think my way out of this box I've thunk
> myself into.
Dave,
This post is going to be direct. Your post sounds like you want/need
a fresh perspective. It is going to be blunt. It is going to even
overstate the case in somewhat simplistic terms to make a key point.
The problem is not re-optimization. The problem is your (mistaken)
conception of what optimization is all about.. So listen up.
> I'm not trying to be argumentative, honest. I
> just don't see any fundamental reason why
> reoptimizing shouldn't work that doesn't
> invalidate the whole idea of backtesting.
There are good ways to optimize and bad ways. If you optimize
the "bad" way, then re-optimizing frequently will still give "bad"
(untrustworthy results). In fact, re-optimization may make a bad
optimization approach give even worse results.
Here is a simple rule of thumb. (I have other rules besides this one
to avoid bad optimization, but this one will illustrate the problem,
if I understand it correctly). I do not optimize to find the "best"
(ie, most profitable) setting. Instead I optimize to find a variable
that gives good results across a RANGE of values. Then I pick one of
the values near the MIDDLE of that range (not necessarily precisely
at the middle, but near the middle). If one wants a robust system
that will likely work in the future, one should avoid use a value
near its edge, even if the value near the edge had the best returns
over the recent past.
For example, if I test a moving average cross system and see good
results with the longer moving average ranging from 10 to 20, one
could decide to use 15. Even if 19 had a much better return, I would
still consider it wise to pick 15 (or 14 or 16) because if the
market behavior moves a bit the range of good values next year might
be 9 to 17 and that would leave the 19 with a poor year. Now if this
were re-optimized after that year I might discover 10 had the best
return over the previous 2 years, but if I picked 10 to trade, the
year after that might see the good range shift back to 11 to 21 and
again I would have a poor year.
Do you see the potential problem here? The problem is not
really "reoptimization" -- the problem was picking the "best"
instead of a more "robust" value. Re-optimization just made a bad
approach worse. The problem is not re-optimization, because if I re-
optimized with the goal of picking one of the 3 middle values, I
would still end up with a "good" but not necessarily the "best"
value for the coming year. In fact, I would not expect re-
optimization to result in chancing the value of choice -- because 15
would still have be near the middle of the good cluster in each re-
optimization.
If you take the time to learn how to take a better approach to
optimization, I think you will find 2 things:
1. Re-optimization will no longer make a system worse.
2. Re-optimization, at least frequent re-optimization, will be
unnecessary. In fact, frequent re-optimization will be a waste of
time because if the original optimization was done properly re-
optimizations should confirm rather than change the settings one is
using. There is a place for occasional re-optimization.
Do you see why #2 would be true? If you do, you are will on the way
to getting out the "box" you mention.
You might wish to read a less pointed post on this issue by Al V.
http://groups.yahoo.com/group/amibroker/message/50303
All the best,
b
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Rent DVDs Online - Over 14,500 titles.
No Late Fees & Free Shipping.
Try Netflix for FREE!
http://us.click.yahoo.com/vhSowB/XP.FAA/3jkFAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|