[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: how to improve generalization in system optimization


  • To: metastock@xxxxxxxxxxxxx
  • Subject: Re: how to improve generalization in system optimization
  • From: lists@xxxxxxxxxxx (rudolf stricker)
  • Date: Mon, 17 Jan 2000 13:59:10 -0800
  • In-reply-to: <LOBBLLPNIJFDCJKGEONHEEEDCCAA.ibarra10@xxxxxxxxxxx>

PureBytes Links

Trading Reference Links

(forwarded on the authority of Angel Ibarra by rudolf stricker)

Dear Rudolf,
I am happy that you find interesting my results. Just a few comments
in the text:

>... I have made some test in
>which I compare the results obtained optimizing a period of time of 10 years
>(in-sample Metastock type of optimization) with the results obtained using a
>"dinamic optimization" each week and applying the obtained parameters for
>the following period. Te comparison between the "real" sistem and the
>in-sample one was differences in earnings between o to 50%. SO I think that
>a value around 75% can be reasonable.

I am afraid I did not explain correctly. I was not talking about the
comparison of the earnings in-sample vs earnings out-of-sample, but I
was talking about other thing. Let me try again: I am trying the
compare the results of the Metastock optimization with a "real"
behaviour...
1) First I optimize a big period of time (for example from 1-1-90 to
1-1-99) and I get a optimal parameter and the total earnings
2) Secondly I optimize a smaller period of time (for example from
1-1-88 to 1-1-90) and I get an optimal parameter
3) I apply this optimal paremet for during a some time (for example
from 1-1-90 to 1-2-90) and I get the earnings for this small period of
time
4) I repeat the optimization for the period of time 1-2-88 to 1-2-90
and I get, again other optimal parameter
5) I apply the optimal parameter for the following period of time
(1-2-90 to 1-3-90) and I get teh earnings for this period of time
6) Sum the earnings of step 3) and 5)
And so on until I reach the 1-1-99. The comparison between the
earnings obtained using "Metastock optimizacion" and "real
optimization" is the ona I made, and I get that the results were
between 0% to 50% lower in the second case, as compared with the first
one.

>My results, seemingly, are not so good as yours: For different types
>of trades (calls/puts, long/short) that I handle in separate trading
>systems, I got differences between the earnings from in-sample and
>out-of-sample tests of about 25 to more than 100%, i.e. profits were
>turned into losses when switching from in-sample to out-of-sample tests.

In relation with this, it is not so surprising. Probably yourin-sample
period is much longer that your out-sample period of time and you
shouldtake in account that probably sometime in your in-sample period
the earningsgoes to cero or lower... I think that the proper
comparison should be made grafically: plot in x-axis the earnings in
the in-sample period and in the y-axis the earnings in the
out-of-sample period. Probably you will get a cloud of points, but it
the cloud is slitghly deformed along the diagonal,
you have detected a significant parameter. If this is not the case,
probable the system has no real meaning

>I have made some test that seems to improve the results of my dinamic
>optimizacion methods. I did not finish the tests but if you want we can
>discuss more in detail. The idea is instead of optimize using only the
>recent past data of the stock (or the index), to use a shorter period of
>time but using data of more similar stocks (in fact I used all the market).
>It is something like using a "market optimization" instead of a "stock"
>optimizacion. With this I obtained also a more stable behaviour of the
>parameters of the system.

>Do I get it right? Are you optimizing the system to the short time
>tendency of the (complete) market, i.e. you design a "local" system
>for the appropriate behavior of the complete market over time and use
>it for special stocks? - Very interesting approach ... - What size of
>short time frame do you use?

That is rigth. When I made optimization with only one stock, I use
data from more or less two years, when I me a "market optimization" I
use something between 6 months to one year data (daily bars only).
Anyhow this is a topic I am only starting with...


>Anyhow I
>believe this should be a function of the system you are using. It should
>depend on the "velocity" of the system.

>Does this "velocity" relate to the volatility of the stock traded or
>to its "basic frequency" e.g. in terms of spectral analysis? - Any hint?

No, when I refer to "velocity" I mean something related to the length
of the trade. I think if you are making trades of a mean lenght of two
months and based in indicators that requires a year of data, you will
need for the optimization period and frequency a much longer period
than if you are using a system with trades of a mean length of two
days and with indicators that use only a week data....


>Nearly the same than my systems, based on continuous ROCs.
>BTW: Imo the "high-sofisticated indicators" are not so important for
>setting up valuable trading systems, i.e. programs like MetaStock imo
>should support much more adequate system optimization capabilities
>rather than support an endless stream of "new" indicators.

I agree with you. The only thing that I think it would be very
important (and I made using VB) is to be able to change the periods of
the indicators as a function of other parameter on a daily basis).
Regards
Angel

| Disclaimer: The views of this user are strictly his own.