[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: how to improve generalization in system optimization


  • To: metastock@xxxxxxxxxxxxx
  • Subject: Re: how to improve generalization in system optimization
  • From: lists@xxxxxxxxxxx (rudolf stricker)
  • Date: Thu, 13 Jan 2000 09:58:30 -0800
  • In-reply-to: <LOBBLLPNIJFDCJKGEONHEEEDCCAA.ibarra10@xxxxxxxxxxx>

PureBytes Links

Trading Reference Links


Dear Angel Ibarra,

thank you much for your comments on my generalization topic!

On Mon, 10 Jan 2000 18:41:03 +0100, you wrote:

>In spite of the interest of your questions regarding the optimization
>problems, I have not seen any answer. I hope you received a lot of
>information as private mails....-).

Sorry, no, unfortunately not ...B-)...

>... I have made some test in
>which I compare the results obtained optimizing a period of time of 10 years
>(in-sample Metastock type of optimization) with the results obtained using a
>"dinamic optimization" each week and applying the obtained parameters for
>the following period. Te comparison between the "real" sistem and the
>in-sample one was differences in earnings between o to 50%. SO I think that
>a value around 75% can be reasonable.

My results, seemingly, are not so good as yours: For different types
of trades (calls/puts, long/short) that I handle in separate trading
systems, I got differences between the earnings from in-sample and
out-of-sample tests of about 25 to more than 100%, i.e. profits were
turned into losses when switching from in-sample to out-of-sample
tests.

This effect turned out to get smaller, when I increased the parameters
responsible for "consistency of wins" (i.e. number of wins/losses,
shape of loss distribution, etc.), but I'm not sure, that the amount
of profit can be kept above zero in all situations, i.e. time frames
of reasonable size.

Therefore, generalization of system optimization, imo is a very
important issue, and I would like to see some appropriate capabilities
to support out-of-sample testing in MetaStock.

>I have made some test that seems to improve the results of my dinamic
>optimizacion methods. I did not finish the tests but if you want we can
>discuss more in detail. The idea is instead of optimize using only the
>recent past data of the stock (or the index), to use a shorter period of
>time but using data of more similar stocks (in fact I used all the market).
>It is something like using a "market optimization" instead of a "stock"
>optimizacion. With this I obtained also a more stable behaviour of the
>parameters of the system.

Do I get it right? Are you optimizing the system to the short time
tendency of the (complete) market, i.e. you design a "local" system
for the appropriate behavior of the complete market over time and use
it for special stocks? - Very interesting approach ... - What size of
short time frame do you use?

BTW: This would remind me of my "weighting function" along time, which
is used for all of my optimization criteria and runs exponentially
between values of e.g. 1 and 7 for oldest to newest trades.

>I also made a study changing the periodicity of the optimization. I obtained
>(more or less) that there is no improvement if you made a lot of
>optimizations. For daily bars, there was no advantage using weekly, monthly
>or even quaterly optimizations. Longer periods of time was worse. 

I made a similar experience, even if I could not figure out something
like an "exact boundary" for the minimum out-of-sample optimization
time frame: For my systems weekly re-optimization seems to make sense
in many cases.

>Anyhow I
>believe this should be a function of the system you are using. It should
>depend on the "velocity" of the system.

Does this "velocity" relate to the volatility of the stock traded or
to its "basic frequency" e.g. in terms of spectral analysis? - Any
hint?

>All the work I have made for the moment, is made using trend following
>systems (something based on the moving averages) and a reduced number of
>stock of the spanish markert.

Nearly the same than my systems, based on continuous ROCs. 
BTW: Imo the "high-sofisticated indicators" are not so important for
setting up valuable trading systems, i.e. programs like MetaStock imo
should support much more adequate system optimization capabilities
rather than support an endless stream of "new" indicators.

>I hope, some of these ideas, not very clearly explained, can be useful for
>you.

Really, they are! - And hopefully, some additional trading system
developers will join this conversation.

Thank you again for your input and comments, Angel!

mfg rudolf stricker
| Disclaimer: The views of this user are strictly his own.