Re: [amibroker] SPSO vs Trib vs CMAE, was: random optimization?, AmiBroker Email List Archive

Hi Ton - The 2 MA crossover system was just a simple example for illustration purposes. In real life I would do an exhaustive opt on that one since it would only have maybe 100 x 100 = 10,000 combinations, and perhaps the small optimization space is why CMAE was able to find the peak. The systems I test with the IO engines generally have at least millions of possible combos and some up into the trillions and more. FWIW, I have done lots of these tests and I will have to stand by my earlier remarks because that is my honest experience, but perhaps others may see different results...

Steve

----- Original Message -----

From: Ton Sieverding

To: amibroker@xxxxxxxxxps.com

Sent: Tuesday, February 03, 2009 3:40 PM

Subject: Re: [amibroker] SPSO vs Trib vs CMAE, was: random optimization?

Interesting ...

..... results, then ran lots of IO tests and compared them to the exhaustive
results to see what the IO's found and also what they missed. You could say
that CMAE seems to take the "safe" approach, IMHO it finds the broad
plateaus pretty well but as you might guess they are usually far from the
most profitable. In my experience, the other two IO engines will generally
find those too but they also find a lot of the smaller and more profitable
ones, which you can then run a mini exhaustive opt on to get a more complete
picture ....

Is that true ? Does CMAE really take the 'safe' approach ? Look to following

picture and see what CMAE gave me as an optimum ...

I got the left peak and hoped to get the plateau in the middle ...

Regards, Ton.

----- Original Message -----

From: Steve Dugas

To: amibroker@xxxxxxxxxps.com

Sent: Tuesday, February 03, 2009 7:54 PM

Subject: Re: [amibroker] SPSO vs Trib vs CMAE, was: random optimization?

Hi Steve - Once you have done an IO and found some results that look
promising, then you can run a mini exhaustive opt if you want. For a simple
example, you run an IO on a MA crossover system, testing both MA's with
periods from 1 to 100. You won't see all possible combos reported but maybe
the results show that MA1=10 and MA2=20 might be good. So to see all the
other values in that neighborhood you could then run a little exhaustive
opt, say MA1 = 5 thru 15 and MA2 = 15 thru 25, something like that, which
will run in a reasonable time.

To test the built-in IO engines, I ran a few exhausive opts and saved the
results, then ran lots of IO tests and compared them to the exhaustive
results to see what the IO's found and also what they missed. You could say
that CMAE seems to take the "safe" approach, IMHO it finds the broad
plateaus pretty well but as you might guess they are usually far from the
most profitable. In my experience, the other two IO engines will generally
find those too but they also find a lot of the smaller and more profitable
ones, which you can then run a mini exhaustive opt on to get a more complete
picture.

Regarding the trade-off you mentioned, I would think it is a matter of
personal taste. How greedy are you? 8 - ) How risk-averse? I am
inclined to try the smaller and higher plateaus first, as long as they have
a little play on each side and are doing well right now, and knowing that
they will fail eventually and I need to keep a close eye on them... Good
luck!

Steve

----- Original Message -----
From: "Steve Davis" <_sdavis@xxxxxxcom>
To: <amibroker@xxxxxxxxxps.com>
Sent: Monday, February 02, 2009 5:01 PM
Subject: [amibroker] SPSO vs Trib vs CMAE, was: random optimization?

> Steve,
>
> I would like to hear more about your system optimization process. How
> were you able to determine the size of the plateaus discovered by the
> built-in optimizers, and how did you decide which solutions had the
> best trade-off between plateau size and profitability?
>
> Thanks,
> another Steve
>
> --- In amibroker@xxxxxxxxxps.com, "Steve Dugas" <sjdugas@xxx> wrote:
>>
>> Hi - I have spent lots of time playing with the built-in intelligent
>> optimizers, in my experience SPSO will return the same results every
> time if
>> the settings are the same. Trib and CMAE will probably return different
>> results each time. FWIW, I find CMAE to be the worst of the three and I
>> don't use it anymore, it will find plateaus but nearly always misses
> the
>> much more profitable but smaller plateaus. Using a quad-core I can
> run 4
>> simultaneous instances and I find that by running 1 SPSO and 3
> Trib's and
>> then comparing the 4 results together, it will generally point me to
> some
>> pretty good param values. Good luck!
>>
>> Steve
>>
>> ----- Original Message -----
>> From: "gabriel_id@..." <finance@xxx>
>> To: <amibroker@xxxxxxxxxps.com>
>> Sent: Monday, February 02, 2009 7:25 AM
>> Subject: [amibroker] Re: random optimization?
>>
>>
>> > OK..
>> >
>> > Can u give me what type of engine and with what kind of settings will
>> > get the same results when i optimize this lines:
>> >
>> > N = Optimize("N-minutes", 33, 1, 60, 1);
>> > TimeFrameSet( N * in1Minute );
>> > MA1 = MA( Close, 10);
>> > MA2 = MA( Close, 20);
>> > BuySignal = Cross( MA1, MA2);
>> > sellSignal = Cross( MA2, MA1);
>> > TimeFrameRestore();
>> >
>> > Buy = TimeFrameExpand(BuySignal , N*in1Minute);
>> > Sell = TimeFrameExpand(sellSignal , N*in1Minute);
>> >
>> > I tried cmae, 5 , 1000, have variable results.. on walkforward
>> > i tried spso, 5, 1000, same variables results..
>> > and also trib, 5, 1000..
>> >
>> >
>> > --- In amibroker@xxxxxxxxxps.com, "Mike" <sfclimbers@> wrote:
>> >>
>> >> Tribes is a non exhaustive optimizer, meaning that it does not
>> >> evaluate every possible combination.
>> >>
>> >> As such, it is possible that it will find different "optimal"
>> >> solutions every time, depending on the nature of the surface being
>> >> optimized. For example; If the surface has many similar peaks, it may
>> >> land on a different one each time (local optima) instead of the one
>> >> true optimal solution (global optima).
>> >>
>> >> Try increasing the number of Runs and/or MaxEval. If you have more
>> >> than 2 or 3 optimization variables, 1000 MaxEval is not enough.
>> >>
>> >> http://amibroker.com/guide/h_optimization.html
>> >>
>> >> Mike
>> >>
>> >> --- In amibroker@xxxxxxxxxps.com, "gabriel_id@" <finance@> wrote:
>> >> >
>> >> > hi there,
>> >> >
>> >> > i am a bit confused, i run the same optimization process.. on same
>> >> > data range.. and i got different results each time :)
>> >> >
>> >> > and the engine was trib, 5, 1000...
>> >> >
>> >> > thx,
>> >> > GV
>> >> >
>> >>
>> >
>> >
>> >
>> > ------------------------------------
>> >
>> > **** IMPORTANT ****
>> > This group is for the discussion between users only.
>> > This is *NOT* technical support channel.
>> >
>> > *********************
>> > TO GET TECHNICAL SUPPORT from AmiBroker please send an e-mail
> directly to
>> > SUPPORT {at} amibroker.com
>> > *********************
>> >
>> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
>> > http://www.amibroker.com/devlog/
>> >
>> > For other support material please check also:
>> > http://www.amibroker.com/support.html
>> >
>> > *********************************
>> > Yahoo! Groups Links
>> >
>> >
>> >
>> >
>>
>
>
>
> ------------------------------------
>
> **** IMPORTANT ****
> This group is for the discussion between users only.
> This is *NOT* technical support channel.
>
> *********************
> TO GET TECHNICAL SUPPORT from AmiBroker please send an e-mail directly to
> SUPPORT {at} amibroker.com
> *********************
>
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> For other support material please check also:
> http://www.amibroker.com/support.html
>
> *********************************
> Yahoo! Groups Links
>
>
>
>