[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: Walk-Forward Out of Sample (OOS) Testing



PureBytes Links

Trading Reference Links

But how about doing it the way I described?  I would prefer to
minimize human interaction and maximize automation.  Imagine testing a
system in this manner on all SP500 stocks, for example.

--- In amibroker@xxxxxxxxxxxxxxx, "Fred" <fctonetti@xxxx> wrote:
> This would be simple enough to do if you think about it terms of how 
> many bars you are going to use for a lookback period and then armed 
> with the results of optimization of that period how far ahead from 
> that point in time you are going to trade before you reoptimize.  As 
> Dingo says this could be done with automation or it could all be done 
> interally in AB/AFL/ABTool.  This is a different form of Score and 
> Rank if you will where instead of scoring and ranking individual 
> issues in a portfolio on a rolling basis you are scoring and ranking 
> parameters for your timing system on a rolling basis.  
> 
> --- In amibroker@xxxxxxxxxxxxxxx, "dingo" <dingo@xxxx> wrote:
> > perhaps I can help - contact me privately at dingo at udsnet dot 
> com.
> >  
> > d
> > 
> > -----Original Message-----
> > From: MarkF2 [mailto:feierstein@x...] 
> > Sent: Tuesday, June 17, 2003 4:00 PM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: [amibroker] Walk-Forward Out of Sample (OOS) Testing
> > 
> > 
> > Jayson, I believe Owen wants to use AB for walk-forward, out of 
> sample
> > testing.  I'd love to be able to do this also but lack the 
> programming
> > skills.  Can anyone help? I think this would be incredibly useful!!!
> > 
> > What I'd like to do can, I believe, be done with loops and would go
> > something like this:
> > 
> > Let's say you're testing a simple system on MSFT where price crosses
> > over and under a moving average so you're optimizing only one
> > parameter, length, to keep things simple. The loop code would allow
> > two inputs: 1) # of segments to break data into and 2) # of segments
> > to use for initial optimization.  Let's say you choose 10 and 3.  AB
> > would divide MSFT data into 10 equal segments, then do the initial
> > optimization on the first 3.  It would then take the best performing
> > length and test it on segment #4 (which is the first OOS segment). 
> > Then it would record two things for this and every other OOS 
> segment:
> > 1) the length used and 2)the result (I'd like profit factor).  Then 
> AB
> > would reoptimize the system over segments 1-4 and use the best
> > parameter on segment 5 (the second OOS segment), recording length 
> used
> > and result, then reoptimize over segments 1-5, use the best 
> parameter
> > on segment 6, etc.  
> > 
> > What this would do is automatically tell you how robust your system
> > concept is over the OOS segments (4-10 in this case).
> > 
> > Mark 
> > 
> > 
> > --- In amibroker@xxxxxxxxxxxxxxx, "Jayson" <jcasavant@xxxx> wrote:
> > > Owen,
> > > Am I over simplifying by suggesting you simply use the
> > > From-To range settings in AA?
> > > 
> > > Example choose 12/1/2002-12/25/2002 to optimize then simply move 
> the
> > window
> > > forward 25 days to test the results.......
> > > 
> > > Regards,
> > > Jayson
> > > -----Original Message-----
> > > From: Owen Davies [mailto:owen5819@x...]
> > > Sent: Tuesday, June 17, 2003 2:04 PM
> > > To: amibroker@xxxxxxxxxxxxxxx
> > > Subject: [amibroker] Annoying programming problem: ideas please?
> > > 
> > > 
> > > Active Trader has taken to doing something I've always considered
> > BS.  Some
> > > articles--at least one a month of late--optimize an intraday 
> trading
> > system,
> > > usually some variation on a volatility breakout, over a month or 
> so
> > of data
> > > and then do an out-of-sample test for the next month or so to
> > "prove" how
> > > well it works.  Over time, this has come to irritate me enough 
> that
> > I'd like
> > > to do a long-term study, either to prove that it's nonsense or to
> > learn
> > > something new and highly unlikely.  So:
> > > 
> > > Can anyone think of a way to optimize a technique on, say, 25 bars
> > of data,
> > > test it on the next 25 bars, and then step the window forward and
> > do it
> > > again?  I'd settle for re-optimizing daily on the previous 25 (or
> > whatever)
> > > bars.  Whatever is easiest.
> > > 
> > > For the sake of simplicity, something that runs on EOD data will 
> do.
> > I mean
> > > to test breakouts in the direction of an existing trend and close 
> at
> > the end
> > > of the day, so there should be no problem with days that break out
> > in one
> > > direction, reverse, and break out in the other; a day that 
> reverses
> > and ends
> > > badly will just count as a loss.  Also, I can do the conversion to
> > intraday
> > > data myself, rather than ask others to hand me the complete 
> package.
> > All I
> > > need is some way to optimize on a window.
> > > 
> > > Offhand, I can't see any way to do it within AFL, and I don't have
> > the skill
> > > to handle it with external programming.
> > > 
> > > Many thanks.
> > > 
> > > Owen Davies
> > > 
> > > 
> > >       Yahoo! Groups Sponsor
> > >             ADVERTISEMENT
> > > 
> > > 
> > > 
> > > 
> > > Send BUG REPORTS to bugs@xxxx
> > > Send SUGGESTIONS to suggest@xxxx
> > > -----------------------------------------
> > > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > --------------------------------------------
> > > Check group FAQ at:
> > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > 
> > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of 
> Service.
> > 
> > 
> > 
> > Yahoo! Groups Sponsor	
> >  
> > 
> <http://rd.yahoo.com/M=251812.3170658.4537139.1261774/D=egroupweb/S=17
> 05
> > 
> 632198:HM/A=1564416/R=0/SIG=11ti81skc/*http://www.netflix.com/Default?
> mq
> > so=60164797&partid=3170658> 	
> >  
> > <http://us.adserver.yahoo.com/l?
> M=251812.3170658.4537139.1261774/D=egrou
> > pmail/S=:HM/A=1564416/rand=707349073> 	
> > 
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
> > 
> > Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service
> > <http://docs.yahoo.com/info/terms/> .


------------------------ Yahoo! Groups Sponsor ---------------------~-->
Get A Free Psychic Reading! Your Online Answer To Life's Important Questions.
http://us.click.yahoo.com/Lj3uPC/Me7FAA/ySSFAA/GHeqlB/TM
---------------------------------------------------------------------~->

Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/