|
PureBytes Links
Trading Reference Links
|
Optimizing has been discussed at several times on this list, but never have
I seen a check-list that could help you identify over optimizing. I welcome
additions to the small list below. If we can identify degrees of
over-optimization we can use the same tool to rate robustness, this because
they are somewhat complementary.
If any of the following statements are True you may be over-optimizing:
1) The system works on only one stock
2) The gains are erratic with a final big gain
3) Optimization values are scattered (random peaks on a 3D chart)
4) Gains are too good to be true
5) The equity line is flat
6) The system only works in recent years
7) It doesn't work out-of-sample (I use 8y dev, 2y out of sample testing)
8) Optimizing parameters vary widely for different time periods
9) Greater than 75% winning trades
7) ... what can you add?
It seem that if you overlay the equities for many systems many appear to
fail for the same periods (use 10 year data), I take this as a sign that the
system pick up on a true market anomaly. It seems to indicates that there
are times that the market is incoherent (chaos rules) and that most systems
will not perform well. Combining equities can produce a "Market Coherence"
(please suggest a better name) indicator which would tell you when trading
is highly risky. Or taking the inverse; a "Market Chaos" indicator. Brave
traders will flip all their signals during those periods :-)
Since there are many parameters one can optimize I take note of the gain
improvement that comes with selectively activating various stages of the
system. I start with the most basic form and add/measure features as I add
them. An unreasonable large jump in gain for any one change makes it suspect
and I will pick it apart manually for a few trades.
|