|
PureBytes Links
Trading Reference Links
|
Dear Angel,
On Mon, 17 Jan 2000 19:50:42 GMT, you wrote:
>1) First I optimize a big period of time (for example from 1-1-90 to
>1-1-99) and I get a optimal parameter and the total earnings
>2) Secondly I optimize a smaller period of time (for example from
>1-1-88 to 1-1-90) and I get an optimal parameter
>3) I apply this optimal paremet for during a some time (for example
>from 1-1-90 to 1-2-90) and I get the earnings for this small period of
>time
>4) I repeat the optimization for the period of time 1-2-88 to 1-2-90
>and I get, again other optimal parameter
>5) I apply the optimal parameter for the following period of time
>(1-2-90 to 1-3-90) and I get teh earnings for this period of time
>6) Sum the earnings of step 3) and 5)
>And so on until I reach the 1-1-99. The comparison between the
>earnings obtained using "Metastock optimizacion" and "real
>optimization" is the ona I made, and I get that the results were
>between 0% to 50% lower in the second case, as compared with the first
>one.
For me this is exactly what I would call a comparison between
in-sample [= your step 1)] and out-of-sample test [= your steps 2) to
6) with "and so on" included]. I did the same using a "sliding time
frame" of 5 years but a weekly update for the systems.
>Probably yourin-sample
>period is much longer that your out-sample period of time and you
>shouldtake in account that probably sometime in your in-sample period
>the earningsgoes to cero or lower...
Even if the size of my sliding window is 5 years and my "extrapolation
horizon" is only one week, the results (collected for a one year
frame) were going bad in some cases, especially for highly volatile
short positions. But I now have made some changes to the rationale of
my systems (in terms of prices taken into account), and hopefully the
results of an out-of-sample test for '99 will turn out positive now,
if I get the final numbers in about two weeks ...
>I think that the proper
>comparison should be made grafically: plot in x-axis the earnings in
>the in-sample period and in the y-axis the earnings in the
>out-of-sample period. Probably you will get a cloud of points, but it
>the cloud is slitghly deformed along the diagonal,
>you have detected a significant parameter. If this is not the case,
>probable the system has no real meaning
That is, you want to see a proportionality between out-of-sample and
in-sample results, which for sure is a valid signal that
"generalization" is working well. But what can we do to _improve_
generalization, at least to some extend? (c.f. the subject)
Denoting a parameter set (e.g. of 5 parameters that I use) as
meaningless, if out-of-sample results are going bad under certain
circumstances sounds a little bit harsh to me, at least if there may
exist some practical measures to enhance generalization.
>That is rigth. When I made optimization with only one stock, I use
>data from more or less two years, when I me a "market optimization" I
>use something between 6 months to one year data (daily bars only).
>Anyhow this is a topic I am only starting with...
I would like to learn from your results, if they point into the
direction of additional generalization capabilities.
Something perhaps comparable that I work with is: My trading systems
use two different time horizons, i.e. short-time for IN/OUT timing and
a somewhat longer time frame to reflect "market behavior", which
influences the validity of the short-time signals. This way, the
IN/OUT timing is adapted to the market behavior.
>I agree with you. The only thing that I think it would be very
>important (and I made using VB) is to be able to change the periods of
>the indicators as a function of other parameter on a daily basis).
Imo, the system _optimization_ capabilities should be very much better
in MetaStock, starting at tests with "constant investment per trade"
(which imo is a must for basic generalization capabilities) up to more
powerful optimization routines, like genetic algorithms, which are
really needed, if we have (highly nonlinear) systems with 5 parameters
or more.
mfg rudolf stricker
PS: All the best for your broken rib ...
| Disclaimer: The views of this user are strictly his own.
|