[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: Walk-Forward Out of Sample (OOS) Testing



PureBytes Links

Trading Reference Links

This would be simple enough to do if you think about it terms of how 
many bars you are going to use for a lookback period and then armed 
with the results of optimization of that period how far ahead from 
that point in time you are going to trade before you reoptimize.  As 
Dingo says this could be done with automation or it could all be done 
interally in AB/AFL/ABTool.  This is a different form of Score and 
Rank if you will where instead of scoring and ranking individual 
issues in a portfolio on a rolling basis you are scoring and ranking 
parameters for your timing system on a rolling basis.  

--- In amibroker@xxxxxxxxxxxxxxx, "dingo" <dingo@xxxx> wrote:
> perhaps I can help - contact me privately at dingo at udsnet dot 
com.
>  
> d
> 
> -----Original Message-----
> From: MarkF2 [mailto:feierstein@x...] 
> Sent: Tuesday, June 17, 2003 4:00 PM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: [amibroker] Walk-Forward Out of Sample (OOS) Testing
> 
> 
> Jayson, I believe Owen wants to use AB for walk-forward, out of 
sample
> testing.  I'd love to be able to do this also but lack the 
programming
> skills.  Can anyone help? I think this would be incredibly useful!!!
> 
> What I'd like to do can, I believe, be done with loops and would go
> something like this:
> 
> Let's say you're testing a simple system on MSFT where price crosses
> over and under a moving average so you're optimizing only one
> parameter, length, to keep things simple. The loop code would allow
> two inputs: 1) # of segments to break data into and 2) # of segments
> to use for initial optimization.  Let's say you choose 10 and 3.  AB
> would divide MSFT data into 10 equal segments, then do the initial
> optimization on the first 3.  It would then take the best performing
> length and test it on segment #4 (which is the first OOS segment). 
> Then it would record two things for this and every other OOS 
segment:
> 1) the length used and 2)the result (I'd like profit factor).  Then 
AB
> would reoptimize the system over segments 1-4 and use the best
> parameter on segment 5 (the second OOS segment), recording length 
used
> and result, then reoptimize over segments 1-5, use the best 
parameter
> on segment 6, etc.  
> 
> What this would do is automatically tell you how robust your system
> concept is over the OOS segments (4-10 in this case).
> 
> Mark 
> 
> 
> --- In amibroker@xxxxxxxxxxxxxxx, "Jayson" <jcasavant@xxxx> wrote:
> > Owen,
> > Am I over simplifying by suggesting you simply use the
> > From-To range settings in AA?
> > 
> > Example choose 12/1/2002-12/25/2002 to optimize then simply move 
the
> window
> > forward 25 days to test the results.......
> > 
> > Regards,
> > Jayson
> > -----Original Message-----
> > From: Owen Davies [mailto:owen5819@x...]
> > Sent: Tuesday, June 17, 2003 2:04 PM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: [amibroker] Annoying programming problem: ideas please?
> > 
> > 
> > Active Trader has taken to doing something I've always considered
> BS.  Some
> > articles--at least one a month of late--optimize an intraday 
trading
> system,
> > usually some variation on a volatility breakout, over a month or 
so
> of data
> > and then do an out-of-sample test for the next month or so to
> "prove" how
> > well it works.  Over time, this has come to irritate me enough 
that
> I'd like
> > to do a long-term study, either to prove that it's nonsense or to
> learn
> > something new and highly unlikely.  So:
> > 
> > Can anyone think of a way to optimize a technique on, say, 25 bars
> of data,
> > test it on the next 25 bars, and then step the window forward and
> do it
> > again?  I'd settle for re-optimizing daily on the previous 25 (or
> whatever)
> > bars.  Whatever is easiest.
> > 
> > For the sake of simplicity, something that runs on EOD data will 
do.
> I mean
> > to test breakouts in the direction of an existing trend and close 
at
> the end
> > of the day, so there should be no problem with days that break out
> in one
> > direction, reverse, and break out in the other; a day that 
reverses
> and ends
> > badly will just count as a loss.  Also, I can do the conversion to
> intraday
> > data myself, rather than ask others to hand me the complete 
package.
> All I
> > need is some way to optimize on a window.
> > 
> > Offhand, I can't see any way to do it within AFL, and I don't have
> the skill
> > to handle it with external programming.
> > 
> > Many thanks.
> > 
> > Owen Davies
> > 
> > 
> >       Yahoo! Groups Sponsor
> >             ADVERTISEMENT
> > 
> > 
> > 
> > 
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > 
> > Your use of Yahoo! Groups is subject to the Yahoo! Terms of 
Service.
> 
> 
> 
> Yahoo! Groups Sponsor	
>  
> 
<http://rd.yahoo.com/M=251812.3170658.4537139.1261774/D=egroupweb/S=17
05
> 
632198:HM/A=1564416/R=0/SIG=11ti81skc/*http://www.netflix.com/Default?
mq
> so=60164797&partid=3170658> 	
>  
> <http://us.adserver.yahoo.com/l?
M=251812.3170658.4537139.1261774/D=egrou
> pmail/S=:HM/A=1564416/rand=707349073> 	
> 
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
> 
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service
> <http://docs.yahoo.com/info/terms/> .


------------------------ Yahoo! Groups Sponsor ---------------------~-->
Get A Free Psychic Reading! Your Online Answer To Life's Important Questions.
http://us.click.yahoo.com/Lj3uPC/Me7FAA/ySSFAA/GHeqlB/TM
---------------------------------------------------------------------~->

Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/