|
PureBytes Links
Trading Reference Links
|
Presactly ...
One of the problems !? for most is that checking for robustness takes
longer. Eons ago Herman posted some fairly decent tools for this
which are along the lines of my own likings i.e. surface area plots.
This is pretty simple to do with systems that have two variables that
are being optimized on and requires more effort for more parameters
but is still doable. You can look at CAR or MDD or MAR or whatever
you like this way and get a good picture of where the cliffs are.
This shows pretty quickly where the most robust i.e. most likely to
succeed points ARE as opposed to the where the most profitable points
WERE. It takes longer then just taking a gander at a list of numbers
but imho it's worth the trip.
I liken the usage of where the most profitable points WERE as chasing
a constantly moving train that you'll never catch up to.
--- In amibroker@xxxxxxxxxxxxxxx, "b519b" <b519b@xxxx> wrote:
> --- "Dave Merrill" <dmerrill@xxxx> wrote:
> > I've thought about all this a great deal,
> > and I really do wish somebody would help
> > me think my way out of this box I've thunk
> > myself into.
>
> Dave,
>
> This post is going to be direct. Your post sounds like you
want/need
> a fresh perspective. It is going to be blunt. It is going to even
> overstate the case in somewhat simplistic terms to make a key
point.
>
> The problem is not re-optimization. The problem is your (mistaken)
> conception of what optimization is all about.. So listen up.
>
> > I'm not trying to be argumentative, honest. I
> > just don't see any fundamental reason why
> > reoptimizing shouldn't work that doesn't
> > invalidate the whole idea of backtesting.
>
> There are good ways to optimize and bad ways. If you optimize
> the "bad" way, then re-optimizing frequently will still give "bad"
> (untrustworthy results). In fact, re-optimization may make a bad
> optimization approach give even worse results.
>
> Here is a simple rule of thumb. (I have other rules besides this
one
> to avoid bad optimization, but this one will illustrate the
problem,
> if I understand it correctly). I do not optimize to find the "best"
> (ie, most profitable) setting. Instead I optimize to find a
variable
> that gives good results across a RANGE of values. Then I pick one
of
> the values near the MIDDLE of that range (not necessarily precisely
> at the middle, but near the middle). If one wants a robust system
> that will likely work in the future, one should avoid use a value
> near its edge, even if the value near the edge had the best returns
> over the recent past.
>
> For example, if I test a moving average cross system and see good
> results with the longer moving average ranging from 10 to 20, one
> could decide to use 15. Even if 19 had a much better return, I
would
> still consider it wise to pick 15 (or 14 or 16) because if the
> market behavior moves a bit the range of good values next year
might
> be 9 to 17 and that would leave the 19 with a poor year. Now if
this
> were re-optimized after that year I might discover 10 had the best
> return over the previous 2 years, but if I picked 10 to trade, the
> year after that might see the good range shift back to 11 to 21 and
> again I would have a poor year.
>
> Do you see the potential problem here? The problem is not
> really "reoptimization" -- the problem was picking the "best"
> instead of a more "robust" value. Re-optimization just made a bad
> approach worse. The problem is not re-optimization, because if I re-
> optimized with the goal of picking one of the 3 middle values, I
> would still end up with a "good" but not necessarily the "best"
> value for the coming year. In fact, I would not expect re-
> optimization to result in chancing the value of choice -- because
15
> would still have be near the middle of the good cluster in each re-
> optimization.
>
> If you take the time to learn how to take a better approach to
> optimization, I think you will find 2 things:
>
> 1. Re-optimization will no longer make a system worse.
>
> 2. Re-optimization, at least frequent re-optimization, will be
> unnecessary. In fact, frequent re-optimization will be a waste of
> time because if the original optimization was done properly re-
> optimizations should confirm rather than change the settings one is
> using. There is a place for occasional re-optimization.
>
> Do you see why #2 would be true? If you do, you are will on the way
> to getting out the "box" you mention.
>
> You might wish to read a less pointed post on this issue by Al V.
> http://groups.yahoo.com/group/amibroker/message/50303
>
> All the best,
>
> b
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Rent DVDs from home.
Over 14,500 titles. Free Shipping
& No Late Fees. Try Netflix for FREE!
http://us.click.yahoo.com/ybSovB/hP.FAA/3jkFAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|