[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: Walk-Forward Out of Sample (OOS) Testing



PureBytes Links

Trading Reference Links

I can't say as I see any benefit in anchoring the front data point as 
this causes current data to be less and less important as time goes 
along.  In addition I don't see why with a properly written AFL there 
would be any need for human intervention.  Basically you supply the 
system and it does the rest much like the way PortfolioTrader that I 
posted uses a user supplied scoring routine to determine which 
securities to invest this would need to decide what parameter values 
to use as time rolls along.

--- In amibroker@xxxxxxxxxxxxxxx, "MarkF2" <feierstein@xxxx> wrote:
> But how about doing it the way I described?  I would prefer to
> minimize human interaction and maximize automation.  Imagine 
testing a
> system in this manner on all SP500 stocks, for example.
> 
> --- In amibroker@xxxxxxxxxxxxxxx, "Fred" <fctonetti@xxxx> wrote:
> > This would be simple enough to do if you think about it terms of 
how 
> > many bars you are going to use for a lookback period and then 
armed 
> > with the results of optimization of that period how far ahead 
from 
> > that point in time you are going to trade before you reoptimize.  
As 
> > Dingo says this could be done with automation or it could all be 
done 
> > interally in AB/AFL/ABTool.  This is a different form of Score 
and 
> > Rank if you will where instead of scoring and ranking individual 
> > issues in a portfolio on a rolling basis you are scoring and 
ranking 
> > parameters for your timing system on a rolling basis.  
> > 
> > --- In amibroker@xxxxxxxxxxxxxxx, "dingo" <dingo@xxxx> wrote:
> > > perhaps I can help - contact me privately at dingo at udsnet 
dot 
> > com.
> > >  
> > > d
> > > 
> > > -----Original Message-----
> > > From: MarkF2 [mailto:feierstein@x...] 
> > > Sent: Tuesday, June 17, 2003 4:00 PM
> > > To: amibroker@xxxxxxxxxxxxxxx
> > > Subject: [amibroker] Walk-Forward Out of Sample (OOS) Testing
> > > 
> > > 
> > > Jayson, I believe Owen wants to use AB for walk-forward, out of 
> > sample
> > > testing.  I'd love to be able to do this also but lack the 
> > programming
> > > skills.  Can anyone help? I think this would be incredibly 
useful!!!
> > > 
> > > What I'd like to do can, I believe, be done with loops and 
would go
> > > something like this:
> > > 
> > > Let's say you're testing a simple system on MSFT where price 
crosses
> > > over and under a moving average so you're optimizing only one
> > > parameter, length, to keep things simple. The loop code would 
allow
> > > two inputs: 1) # of segments to break data into and 2) # of 
segments
> > > to use for initial optimization.  Let's say you choose 10 and 
3.  AB
> > > would divide MSFT data into 10 equal segments, then do the 
initial
> > > optimization on the first 3.  It would then take the best 
performing
> > > length and test it on segment #4 (which is the first OOS 
segment). 
> > > Then it would record two things for this and every other OOS 
> > segment:
> > > 1) the length used and 2)the result (I'd like profit factor).  
Then 
> > AB
> > > would reoptimize the system over segments 1-4 and use the best
> > > parameter on segment 5 (the second OOS segment), recording 
length 
> > used
> > > and result, then reoptimize over segments 1-5, use the best 
> > parameter
> > > on segment 6, etc.  
> > > 
> > > What this would do is automatically tell you how robust your 
system
> > > concept is over the OOS segments (4-10 in this case).
> > > 
> > > Mark 
> > > 
> > > 
> > > --- In amibroker@xxxxxxxxxxxxxxx, "Jayson" <jcasavant@xxxx> 
wrote:
> > > > Owen,
> > > > Am I over simplifying by suggesting you simply use the
> > > > From-To range settings in AA?
> > > > 
> > > > Example choose 12/1/2002-12/25/2002 to optimize then simply 
move 
> > the
> > > window
> > > > forward 25 days to test the results.......
> > > > 
> > > > Regards,
> > > > Jayson
> > > > -----Original Message-----
> > > > From: Owen Davies [mailto:owen5819@x...]
> > > > Sent: Tuesday, June 17, 2003 2:04 PM
> > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > Subject: [amibroker] Annoying programming problem: ideas 
please?
> > > > 
> > > > 
> > > > Active Trader has taken to doing something I've always 
considered
> > > BS.  Some
> > > > articles--at least one a month of late--optimize an intraday 
> > trading
> > > system,
> > > > usually some variation on a volatility breakout, over a month 
or 
> > so
> > > of data
> > > > and then do an out-of-sample test for the next month or so to
> > > "prove" how
> > > > well it works.  Over time, this has come to irritate me 
enough 
> > that
> > > I'd like
> > > > to do a long-term study, either to prove that it's nonsense 
or to
> > > learn
> > > > something new and highly unlikely.  So:
> > > > 
> > > > Can anyone think of a way to optimize a technique on, say, 25 
bars
> > > of data,
> > > > test it on the next 25 bars, and then step the window forward 
and
> > > do it
> > > > again?  I'd settle for re-optimizing daily on the previous 25 
(or
> > > whatever)
> > > > bars.  Whatever is easiest.
> > > > 
> > > > For the sake of simplicity, something that runs on EOD data 
will 
> > do.
> > > I mean
> > > > to test breakouts in the direction of an existing trend and 
close 
> > at
> > > the end
> > > > of the day, so there should be no problem with days that 
break out
> > > in one
> > > > direction, reverse, and break out in the other; a day that 
> > reverses
> > > and ends
> > > > badly will just count as a loss.  Also, I can do the 
conversion to
> > > intraday
> > > > data myself, rather than ask others to hand me the complete 
> > package.
> > > All I
> > > > need is some way to optimize on a window.
> > > > 
> > > > Offhand, I can't see any way to do it within AFL, and I don't 
have
> > > the skill
> > > > to handle it with external programming.
> > > > 
> > > > Many thanks.
> > > > 
> > > > Owen Davies
> > > > 
> > > > 
> > > >       Yahoo! Groups Sponsor
> > > >             ADVERTISEMENT
> > > > 
> > > > 
> > > > 
> > > > 
> > > > Send BUG REPORTS to bugs@xxxx
> > > > Send SUGGESTIONS to suggest@xxxx
> > > > -----------------------------------------
> > > > Post AmiQuote-related messages ONLY to: 
amiquote@xxxxxxxxxxxxxxx
> > > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > > --------------------------------------------
> > > > Check group FAQ at:
> > > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > > 
> > > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of 
> > Service.
> > > 
> > > 
> > > 
> > > Yahoo! Groups Sponsor	
> > >  
> > > 
> > 
<http://rd.yahoo.com/M=251812.3170658.4537139.1261774/D=egroupweb/S=17
> > 05
> > > 
> > 
632198:HM/A=1564416/R=0/SIG=11ti81skc/*http://www.netflix.com/Default?
> > mq
> > > so=60164797&partid=3170658> 	
> > >  
> > > <http://us.adserver.yahoo.com/l?
> > M=251812.3170658.4537139.1261774/D=egrou
> > > pmail/S=:HM/A=1564416/rand=707349073> 	
> > > 
> > > Send BUG REPORTS to bugs@xxxx
> > > Send SUGGESTIONS to suggest@xxxx
> > > -----------------------------------------
> > > Post AmiQuote-related messages ONLY to: 
amiquote@xxxxxxxxxxxxxxx 
> > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > --------------------------------------------
> > > Check group FAQ at:
> > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
> > > 
> > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of 
Service
> > > <http://docs.yahoo.com/info/terms/> .


------------------------ Yahoo! Groups Sponsor ---------------------~-->
Get A Free Psychic Reading! Your Online Answer To Life's Important Questions.
http://us.click.yahoo.com/Lj3uPC/Me7FAA/ySSFAA/GHeqlB/TM
---------------------------------------------------------------------~->

Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/