|
PureBytes Links
Trading Reference Links
|
--- In amibroker@xxxx, "Tomasz Janeczko" <amibroker@xxxx> wrote:
> This brings the question of curve-fitting brought by David.
> The system is very sensitive to the changes of X1, Dbuy, Dsell
> variables.
> The problem is that once we optimize for single stock
> we get "perfect"
> parameters but only for this one stock and the history we just
> used to
> optimize. What is needed is "out-of-sample" testing to
> verify the results.
> ===============
I am far from an expert on optimization/curve fitting, but a few
approaches have been helpful to me.
1. Does it make sense LOGICALLY? I know from experience what trouble
I can get into using logic (this one "should" go up) without
confirmating the method with extensive backtesting. But I think there
is a place for logic. If optimization gives a great result for an
unusal variable, especially if the next nearest variables give poor
results, I have no confidence in the results unless I can find a
logical reason to explain why neighbouring variables could give such
different results.
2. How consistently are the optimized results? One way to do this is
to test the optimized values on "out of range" data (ie, data not
used in the optimization runs). If the results are not similar my
confidence is low. Another way I sometimes confirm optimization
results is to "segment" the time period. I take my optimized values
from say a 10 year period and try them on several 1 year periods
within the 10 years. If an "optimized" value is going to give some
years of negative results, I want to know ahead of time before
putting real money in.
3. I sometimes choose the "safest" value even if it is not
the "best". Consider the following:
Value, gain
1, 10%
2, 15%
3, 18%
4, 20%
5, 8%
6, 5%
Here I would perfer to use value 3 even though slightly better. My
reasoning? If the market patterns shift slightly, 4 could easily
become like 5. The shift would have to be greater to affect the
returns from 3.
4. I am always on the look out for out of pattern results: Consider
1, 5%
2, 10%
3, 20%
4, 5%
5, -2%
6, 30%
7, -3%
8, -5%
Here, I would be very suspicious of the optimised value of 6. I
suspect it is due more to a random data set than to a repeating
market event.
b
|