[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Suggestion for better backtesting/optimization



PureBytes Links

Trading Reference Links

I'm missing something then. I want to do an optimization within an 
optimization. I don't see how I can programmatically (without using 
JScript or VB) select the "best" parameters from an optimization run 
and then use them in a subsequent backtest step, for one day only, 
then repeat that for all days available WITHOUT any manual 
intervention...

What I want to do is (assuming we have, say, 2000 data points, and a 
system that has 2 variables that can be optimized),
find the "best" opt_period:

For opt_period=250 to 1000 by 50
For begDate = 0 to 2000-opt_period by 1
endDate = begDate + opt_period

// Find best opt_var1, opt_var2 for begDate to endDate
For opt_var1 = v1init to v1end by v1step
For opt_var2 = v2init to v2end by v2step
result1[] = backtest from begDate to endDate
End for
End For

// Decide what to do on endDate+1
Select opt_var1, opt_var2 from "best" result1[]
Use chosen opt_var1 and opt_var2 for endDate+1
End For

Save result2[]
End For
list values from result2[]

Larry Lewis


--- In amibroker@xxxx, "Tomasz Janeczko" <amibroker@xxxx> wrote:
> Larry,
> 
> As Al wrote, there is no obstacle in adding additional optimization
> parameter that controls walk-forward test.
> 
> Best regards,
> Tomasz Janeczko
> amibroker.com
> ----- Original Message ----- 
> From: "lel4866" <lel@xxxx>
> To: <amibroker@xxxx>
> Sent: Monday, September 30, 2002 5:30 PM
> Subject: [amibroker] Re: Suggestion for better 
backtesting/optimization
> 
> 
> > This is not what I meant -
> > 
> > I would like to do all of this in an automated fashion where the 
> > look back period and stable parameter period are both 
optimization 
> > variables, and the parameter selection was automatic.
> > 
> > Larry Lewis
> > 
> > --- In amibroker@xxxx, "Avcinci" <avcinci@xxxx> wrote:
> > > You can do walk-forward testing now with AB. Simply optimize 
from, 
> > say, 1997 to 2000, then using the optimum parameter values from 
that 
> > optimization, backtest on the same stocks from 2000 to present 
(out 
> > of sample data). You can vary the time periods to do your 
> > optimizations and backtests. Of course, this is 2 steps, but it's 
> > not that hard. TJ has said that, later this year, he will be 
> > incorporating in his graphics engine 3-D graphics that enable the 
> > user to view the robustness of an optimization in a manner 
similar 
> > to but better than the Excel add-in that Herman developed and 
> > uploaded earlier. You would choose parameter values from a flat 
area 
> > of the optimization graph where the parameter values and equity 
> > curve would not change much from one set to another. 
> > > 
> > > Al V.
> > > ----- Original Message ----- 
> > > From: lel4866 
> > > To: amibroker@xxxx 
> > > Sent: Sunday, September 29, 2002 12:29 PM
> > > Subject: [amibroker] Suggestion for better 
> > backtesting/optimization
> > > 
> > > 
> > > My suggestion is for support for walk forward optimization. 
This 
> > is 
> > > what we all actually do in real life. Since we can't look 
into 
> > the 
> > > future, we test/optimize based on the past, then apply the 
> > results 
> > > to the following days trading. We then look at the results 
and 
> > make 
> > > additional changes.
> > > 
> > > There are 2 simple variables: 1) The length of time we look 
> > back, 
> > > and the length of time (or other conditions) we wait until we 
re-
> > > optimize.
> > > 
> > > There's a 3rd, more complicated variable - how we choose the 
> > > optimium parameters from the last optimization run. My 
> > suggestion is 
> > > for a "score" formula that takes into account things like: 
> > maximum % 
> > > drawdown, best compound rate of return, highest minimum of 
the 
> > > running 1 year returns (or whatever period you like).
> > > 
> > > I particularly like the last one - I always look at a graph 
of 
> > the 1 
> > > year returns (Equity() - Ref(Equity(), -253)). Ideally, it 
> > should be 
> > > as flat as possible (or maybe rising). I've been treating 
this a 
> > > little like analysis of variance - it should look like white 
> > noise 
> > > with a given mean and standard deviation - any periods that 
> > don't 
> > > bear investingation.
> > > 
> > > Right now I use AmiBroker for experiments, but when I find an 
> > idea I 
> > > like, I must write my own program to do walk-forward testing.
> > > 
> > > Larry Lewis
> > > 
> > > 
> > > 
> > > Yahoo! Groups Sponsor 
> > > ADVERTISEMENT
> > > 
> > > 
> > > 
> > > 
> > > Post AmiQuote-related messages ONLY to: amiquote@xxxx 
> > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > 
> > > Check group FAQ at: 
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
> > > 
> > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of 
> > Service.
> > 
> > 
> > 
> > Post AmiQuote-related messages ONLY to: amiquote@xxxx 
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > 
> > Check group FAQ at: 
http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
> > 
> > Your use of Yahoo! Groups is subject to 
http://docs.yahoo.com/info/terms/ 
> > 
> > 
> >