|
PureBytes Links
Trading Reference Links
|
Gary,
As I wrote in my previous message, the method is wrong.
The chain rule is not valid for optimizations
opt(x,y,z) is not opt(x,opt(y,opt(z)))
It is not unusual in mathematics.
The sin(a+b) is not sina+sinb
The f[g(x)]´ is not f[g´(x)].
If, in your example, the first optimal is x=20, then you will never
reach the [90,10,70] solution, no matter how many iterations you
execute with the 25%, 20%, 10%, 5% plus or minus.
You try something like the exaustion method, but we always need a
theoretical background to do it. The existence theorems are always
*before* the exaustion application, else it is useless to apply
exaustion.
If the limit of a function does not exist, then any method to
calculate this limit will fail.
Except this, the whole procedure you describe takes too much time.
Dimitris Tsokakis
PS: For a quick T/A example try the
Plot(EMA(EMA(C,50),10),"",1,1);
Plot(EMA(EMA(C,10),50),"",4,1);
You will see two matching lines.
Then, try the
Plot(Wilders(Wilders(C,50),10),"",1,1);
Plot(Wilders(Wilders(C,10),50),"",4,1);
The lines are pretty identical again.
Here comes the trap:
The result is that we may use a composite MA, EMA or Wilders
transformation, no matter of the periods priority.
Is it always true ? Can we apply the same logic to any T/A function ?
Simply try
Plot(RSIa(RSIa(C,50),10),"",1,1);
Plot(RSIa(RSIa(C,10),50),"",4,1);
and you will see.
I hope it is clear.
--- In amibroker@xxxxxxxxxxxxxxx, "Gary A. Serkhoshian"
<serkhoshian777@xxxx> wrote:
> Dimitris and Herman,
>
> Thank you both for your time, and Herman thanks for the code snipit.
>
> Dimitris, since you were kind enough to demonstrate optimizations
via code, I thought to take my explaination a step further via an
example and show a shortcoming of this iterative process.
>
> I'll summarize, so that the code progression will make sense. I
optimized all three params using Portfolio Optimization and used UPI
as my objective function. This was my bogey that my iterative
process should hit.
>
> Well, as you will see after 7 iterations my UPI actually stabilized
at a level higher than my bogey. How can this be possible? Well, as
you will see below once I have an optimized param I then introduce a
certain degree of randomness for the next iteration. This offerss
the ability to test the robustness of the params to some degree, but
also the ability to take smaller steps in the varaibles
anticipated "sweet spot".
>
> What does this all mean? Well, taking small steps I end up finding
local maxima that steps of 10 miss. BUT THIS IS NOT GOOD NEWS!!!
This iterative process leads us to the top of Mt. Everest with scary,
steep cliffs all around us. This is not where we want out system to
live.
>
> So what do we do? We must go a step further and look at the slope
of the objective function as we walk away from the optimal point.
Consequently we've got to find smoother peaks, and get the hell away
from the cliffs.
>
> This last part of identifying the shallow slope of the objective
function following an iteration is what I'm trying code now.
>
> Code below.
>
> Hope this helps,
> Gary
>
> //Tested on !RUT starting from 01/02/1992
>
> /*
>
> //Optimize all params. This is the bogey we want to hit via
iterative optimizations. Let's see how close we get.
>
> x=Optimize("x",10,10,100,10);
>
> b=Optimize("b",10,10,50,10);
>
> s=Optimize("s",50,50,90,10);
>
> Buy=Cross(StochD(x),b);
>
> Sell=Cross(s,StochD(x));
>
> //Best result based on UPI = 0.72 [x=90, b=10, s=70]
>
> */
>
> //======================= START ITERATIVE OPTIMIZATION BELOW
=====================================
>
> //We will keep iterating until we get a UPI that is within 5% of
last iteration. Even after we find an optimial param we wil still
introduce some randomness in the range that will decrease with each
passing iteration with the idea that we are getting close and close
to our bogey. The randomness will step down 20%,15%,10%,5%,2.5%,0%
range above and below optimal param.
>
> /*
>
> //Iteration 1
>
> x=Optimize("x",10,10,100,10);
>
> b= 10;//Optimize("b",10,10,50,10);
>
> s= 50;//Optimize("s",50,50,90,10);
>
> Buy=Cross(StochD(x),b);
>
> Sell=Cross(s,StochD(x));
>
> //Best result based on UPI = 0.29 [x=90, b=10, s=50]. We have alot
of work ahead of us if we want to get to bogey of 0.72
>
>
>
>
>
> //Iteration 2
>
> x=Optimize("x",90,72,108,1); //20% range above and below optimal
param to introduce randomness
>
> b= Optimize("b",10,10,50,10); //Optimize b over full range
>
> s= 50;//Optimize("s",50,50,90,10);
>
> Buy=Cross(StochD(x),b);
>
> Sell=Cross(s,StochD(x));
>
> //Best result based on UPI = 0.34 [x=94, b=40, s=50]. Still not
close. Note how x has changed modestly as we introduce some randomness
>
>
>
> //Iteration 3
>
> x=Optimize("x",94,80,108,1); //15% range above and below optimal
param to introduce randomness
>
> b= Optimize("b",40,32,48,1); //20% range above and below optimal
param to introduce randomness
>
> s= Optimize("s",50,50,90,10); //Optimize s over full range
>
> Buy=Cross(StochD(x),b);
>
> Sell=Cross(s,StochD(x));
>
> //Best result based on UPI = 0.86 [x=82, b=33, s=60]. We actually
beat the bogey, but remember we need to keep iterating untill the UPI
stabilizes. Otherwise, these numbers are bogus.
>
> //Iteration 4
>
> x=Optimize("x",82,74,90,1); //10% range above and below optimal
param to introduce randomness
>
> b= Optimize("b",33,28,38,1); //15% range above and below optimal
param to introduce randomness
>
> s= Optimize("s",60,48,72,1); //20% range above and below optimal
param to introduce randomness
>
> Buy=Cross(StochD(x),b);
>
> Sell=Cross(s,StochD(x));
>
> //Best result based on UPI = 0.87 [x=74, b=33, s=58]. Okay UPI is
close to last iteration. Let's iterate again and see what happens.
>
>
>
> //Iteration 5
>
> x=Optimize("x",74,70,78,1); //5% range above and below optimal
param to introduce randomness
>
> b= Optimize("b",33,30,36,1); //10% range above and below optimal
param to introduce randomness
>
> s= Optimize("s",58,49,67,1); //15% range above and below optimal
param to introduce randomness
>
> Buy=Cross(StochD(x),b);
>
> Sell=Cross(s,StochD(x));
>
> //Best result based on UPI = 1.11 [x=72, b=33, s=61]. Probably a
pencil-point UPI. Hasn't stablized yet so back to salt mines.
>
> //Iteration 6
>
> x=Optimize("x",72,70,74,1); //2.5% range above and below optimal
param to introduce randomness
>
> b= Optimize("b",33,31,35,1); //5% range above and below optimal
param to introduce randomness
>
> s= Optimize("s",61,55,67,1); //10% range above and below optimal
param to introduce randomness
>
> Buy=Cross(StochD(x),b);
>
> Sell=Cross(s,StochD(x));
>
> //Best result based on UPI = 1.11 [x=72, b=33, s=61]. Okay UPI same
as last iteration. Could this be the light at end of tunnel. Let's
iterate again and see what happens.
>
> */
>
>
>
> //Iteration 7
>
> x=72; //Optimize("x",72,70,74,1); //0% range above and below
optimal param to introduce randomness
>
> b= Optimize("b",33,32,34,1); //2.5% range above and below optimal
param to introduce randomness
>
> s= Optimize("s",61,58,64,1); //5% range above and below optimal
param to introduce randomness
>
> Buy=Cross(StochD(x),b);
>
> Sell=Cross(s,StochD(x));
>
> //Best result based on UPI = 1.11 [x=72, b=33, s=61]. Okay UPI same
as last iteration. But by looking at the optimizatio matrix, we are
on the top of Mt. Everest with very steep cliffs all around us. The
better question is why didn't we get close to the bogey params? How
can we correct this?
>
>
> DIMITRIS TSOKAKIS <TSOKAKIS@xxxx> wrote:
> Herman,
> OK, I see.
> BTW, this is nice for individual optimals...
> Dimitris Tsokakis
> --- In amibroker@xxxxxxxxxxxxxxx, "Herman vandenBergen"
<psytek@xxxx>
> wrote:
> > Hello DT,
> >
> > The procedure I suggested simply cascades inline optimization
loops
> (one
> > stage only) as shown below, carrying forward optimized variables.
I
> just
> > copied this code from my archive without verification but
hopefully
> it
> > conveys the idea.
> >
> > herman
> >
> > // Inline optimization
> > bestequity = 0;
> > bestrange = 0;
> > // optimization loop
> > for( range = 10; range < 20; range ++ )
> > {
> > Buy = Cross( Close, EMA( Close, range ) );
> > Sell = Cross( EMA( Close, range ), Close );
> > LastEquity = LastValue( Equity() );
> > if( LastEquity > bestequity )
> > {
> > bestequity = LastEquity;
> > bestrange = range;
> > }
> > E=Equity(1);
> > AddColumn( BestRange, "R="+WriteVal(range,1.0));
> > AddColumn( BestEquity, "E="+WriteVal(range,1.0));
> > }
> > Range = BestRange;
> > Buy = Cross( Close, EMA( Close, range ) );
> > Sell = Cross( EMA( Close, range ), Close );
> > Filter = Status("LastBarInTest");
> > herman.
> >
> > -----Original Message-----
> > From: DIMITRIS TSOKAKIS [mailto:TSOKAKIS@x...]
> > Sent: November 13, 2003 4:56 PM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: [amibroker] Re: Help on automating optimization (genetic
> > algorithms)
> >
> >
> > Herman,
> > How did you do it ?
> > Because:
> > The
> > x=Optimize("x",10,10,50,10);b=Optimize("b",10,10,50,10);
> > Buy=Cross(StochD(x),b);
> > Sell=Cross(80,StochD(x));Short=Sell;Cover=Buy;
> > needs 25 steps.
> > The
> > x=Optimize("x",10,10,50,10);
> > for(b=10;b<=50;b=b+10)
> > {
> > Buy=Cross(StochD(x),b);
> > Sell=Cross(80,StochD(x));Short=Sell;Cover=Buy;
> > }
> > needs 5 steps, because the results will have *only* the last b=50
> > choice.
> > The
> > for(b=10;b<50;b=b+10)
> > {
> > x=Optimize("x",10,10,50,10);
> > Buy=Cross(StochD(x),b);
> > Sell=Cross(80,StochD(x));Short=Sell;Cover=Buy;
> > }
> > needs 625 [!!] steps and *does not* give the optimal solution.
> > Is it possible to give an example?
> > Dimitris Tsokakis
> > --- In amibroker@xxxxxxxxxxxxxxx, "Steve Dugas" <sjdugas@xxxx>
> wrote:
> > > Hi,
> > >
> > > True, unfortunately. A simple way to look at it is by counting
the
> > number of
> > > combinations tested. For example, say I am optimizing on 2
> > variables, each
> > > with 10 steps. If I optimize them together, I am testing 10 x
10
> or
> > 100
> > > combinations. If I optimize them one at a time, I am testing 10
+
> > 10 or 20
> > > combinatins. The second way will certainly be faster, because I
am
> > only
> > > testing 20% of the possible combinations. The true optimal
results
> > are
> > > likely to lie somewhere in the 80% of combinations that I have
not
> > tested..
> > > The more variables used, the % of possible combinations actually
> > tested
> > > continues to drop. For 3 variables, 30/1000 or only 3%. For 4
> > variables,
> > > 40/10,000 or only 0.4 %, etc...
> > >
> > > Steve
> > >
> > > ----- Original Message -----
> > > From: "DIMITRIS TSOKAKIS" <TSOKAKIS@xxxx>
> > > To: <amibroker@xxxxxxxxxxxxxxx>
> > > Sent: Thursday, November 13, 2003 12:47 AM
> > > Subject: [amibroker] Re: Help on automating optimization
(genetic
> > > algorithms)
> > >
> > >
> > > > Gary,
> > > > The procedure you describe does not make any sense.
> > > > In the simple example
> > > > x=Optimize("x",10,10,100,10);
> > > > b=Optimize("b",10,10,50,10);
> > > > s=Optimize("s",50,50,90,10);
> > > > Buy=Cross(StochD(x),b);
> > > > Sell=Cross(s,StochD(x));
> > > > the optimal solution is [30,20,80]
> > > > If I optimize first the x, ie
> > > > x=Optimize("x",10,10,100,10);
> > > > b=10;//Optimize("b",10,10,50,10);
> > > > s=50;//Optimize("s",50,50,90,10);
> > > > Buy=Cross(StochD(x),b);
> > > > Sell=Cross(s,StochD(x));
> > > > [optimal x=80]
> > > > then the b, ie
> > > > x=80;//Optimize("x",10,10,100,10);
> > > > b=Optimize("b",10,10,50,10);
> > > > s=50;//Optimize("s",50,50,90,10);
> > > > Buy=Cross(StochD(x),b);
> > > > Sell=Cross(s,StochD(x));
> > > > [optimal b=10]
> > > > and finally the s, ie
> > > > x=80;//Optimize("x",10,10,100,10);
> > > > b=10;//Optimize("b",10,10,50,10);
> > > > s=Optimize("s",50,50,90,10);
> > > > Buy=Cross(StochD(x),b);
> > > > Sell=Cross(s,StochD(x));
> > > > [optimal s=90]
> > > > I will have the [80,10,90] solution, which is far from the
> actual
> > > > optimal [30,20,80]
> > > > Unfortunately, the optimal[x,b,s] *is not* the optimal[optimal
> > [optimal
> > > > [x],b],s], consequently this procedure is wrong from the
> > beginning.
> > > > There is no chain rule for composite optimizations and you
will
> > have
> > > > very few probabilities to find the optimal.
> > > > Another basic issue: You do not even have the
> > > > optimal[optimal[x],y] equal to optimal[optimal[y],x]
> > > > property, both will be far from the optimal[x,y] and you can
not
> > even
> > > > interchange the priorities.
> > > >
> > > > Herman,
> > > > The results of cascade optimizations will not give the optimal
> > > > result, even if you save some time. As for this last
property, I
> > do
> > > > not think you save any time, optimization is an internal
loop,
> as
> > far
> > > > as I understand...
> > > > Dimitris Tsokakis
> > > >
> > > >
> > > > --- In amibroker@xxxxxxxxxxxxxxx, "Herman vandenBergen"
> > <psytek@xxxx>
> > > > wrote:
> > > > > You can optimize using for() loops and examples of this
method
> > have
> > > > been
> > > > > posted on the list. This would allow you to optimize one
> > parameter
> > > > first in
> > > > > the first loop-optimization and then go on the the next
loop-
> > > > optimization
> > > > > using the result of the first loop-optimization in your
> > > > calculations. This
> > > > > way you will save lots of time and you can optimize an
> unlimited
> > > > number of
> > > > > parameters without looking at billions of iterations.
> > > > >
> > > > > happy tinkering,
> > > > > herman.
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Gary A. Serkhoshian [mailto:serkhoshian777@x...]
> > > > > Sent: November 13, 2003 7:00 AM
> > > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > > Subject: [amibroker] Help on automating optimization
> (genetic
> > > > algorithms)
> > > > >
> > > > >
> > > > > Hi all,
> > > > >
> > > > > I should preface this e-mail by saying that I'm not
trying
> to
> > > > create a
> > > > > Rube Goldberg as I can go through the process ourlined below
> > > > manually. So,
> > > > > if what I'm suggesting requires a Rube Goldberg type
solution
> > let
> > > > me know,
> > > > > and I'll let it go.
> > > > >
> > > > > Essentially what I'm doing with optimizations over
multiple
> > > > variables is
> > > > > prioritizing variables on importance, and optimizing one
> > variable
> > > > at a time
> > > > > leaving the others static. I just keep iterating through
all
> > the
> > > > variables
> > > > > until things stabilize in terms of standard measures. The
> > cocktail
> > > > party
> > > > > term is genetic algorithms, but between us it's necesary as
> > > > optimizing over
> > > > > multiple variables (4+) will either take too long or worse
> > cause
> > > > AmiBroker
> > > > > to crash due to memory issues.
> > > > >
> > > > > Here's the punchline. Can you all give some input on
ways
> to
> > > > program this
> > > > > in afl so as to automate the process outlined above rather
> than
> > me
> > > > manually
> > > > > iterating through as it is labor intensive.
> > > > >
> > > > > Many thanks,
> > > > > Gary
> > > > >
> > > > >
> > > > > ------------------------------------------------------------
--
> --
> > ----
> > > > --------
> > > > > --
> > > > > Do you Yahoo!?
> > > > > Protect your identity with Yahoo! Mail AddressGuard
> > > > > Send BUG REPORTS to bugs@xxxx
> > > > > Send SUGGESTIONS to suggest@xxxx
> > > > > -----------------------------------------
> > > > > Post AmiQuote-related messages ONLY to:
> > amiquote@xxxxxxxxxxxxxxx
> > > > > (Web page:
http://groups.yahoo.com/group/amiquote/messages/)
> > > > > --------------------------------------------
> > > > > Check group FAQ at:
> > > > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > > >
> > > > > Your use of Yahoo! Groups is subject to the Yahoo! Terms
of
> > > > Service.
> > > >
> > > >
> > > >
> > > > Send BUG REPORTS to bugs@xxxx
> > > > Send SUGGESTIONS to suggest@xxxx
> > > > -----------------------------------------
> > > > Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx
> > > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > > --------------------------------------------
> > > > Check group FAQ at:
> > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > >
> > > > Your use of Yahoo! Groups is subject to
> > http://docs.yahoo.com/info/terms/
> > > >
> > > >
> >
> >
> > Yahoo! Groups Sponsor
> > ADVERTISEMENT
> >
> >
> >
> >
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
Service.
>
>
> Yahoo! Groups SponsorADVERTISEMENT
>
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
http://groups.yahoo.com/group/amibroker/files/groupfaq.html
>
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of
Service.
>
>
> ---------------------------------
> Do you Yahoo!?
> Protect your identity with Yahoo! Mail AddressGuard
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|