[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Walk-Forward Out of Sample (OOS) Testing



PureBytes Links

Trading Reference Links

Jayson, I believe Owen wants to use AB for walk-forward, out of sample
testing.  I'd love to be able to do this also but lack the programming
skills.  Can anyone help? I think this would be incredibly useful!!!

What I'd like to do can, I believe, be done with loops and would go
something like this:

Let's say you're testing a simple system on MSFT where price crosses
over and under a moving average so you're optimizing only one
parameter, length, to keep things simple. The loop code would allow
two inputs: 1) # of segments to break data into and 2) # of segments
to use for initial optimization.  Let's say you choose 10 and 3.  AB
would divide MSFT data into 10 equal segments, then do the initial
optimization on the first 3.  It would then take the best performing
length and test it on segment #4 (which is the first OOS segment). 
Then it would record two things for this and every other OOS segment:
1) the length used and 2)the result (I'd like profit factor).  Then AB
would reoptimize the system over segments 1-4 and use the best
parameter on segment 5 (the second OOS segment), recording length used
and result, then reoptimize over segments 1-5, use the best parameter
on segment 6, etc.  

What this would do is automatically tell you how robust your system
concept is over the OOS segments (4-10 in this case).

Mark 


--- In amibroker@xxxxxxxxxxxxxxx, "Jayson" <jcasavant@xxxx> wrote:
> Owen,
> Am I over simplifying by suggesting you simply use the
> From-To range settings in AA?
> 
> Example choose 12/1/2002-12/25/2002 to optimize then simply move the
window
> forward 25 days to test the results.......
> 
> Regards,
> Jayson
> -----Original Message-----
> From: Owen Davies [mailto:owen5819@x...]
> Sent: Tuesday, June 17, 2003 2:04 PM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: [amibroker] Annoying programming problem: ideas please?
> 
> 
> Active Trader has taken to doing something I've always considered
BS.  Some
> articles--at least one a month of late--optimize an intraday trading
system,
> usually some variation on a volatility breakout, over a month or so
of data
> and then do an out-of-sample test for the next month or so to
"prove" how
> well it works.  Over time, this has come to irritate me enough that
I'd like
> to do a long-term study, either to prove that it's nonsense or to
learn
> something new and highly unlikely.  So:
> 
> Can anyone think of a way to optimize a technique on, say, 25 bars
of data,
> test it on the next 25 bars, and then step the window forward and
do it
> again?  I'd settle for re-optimizing daily on the previous 25 (or
whatever)
> bars.  Whatever is easiest.
> 
> For the sake of simplicity, something that runs on EOD data will do.
 I mean
> to test breakouts in the direction of an existing trend and close at
the end
> of the day, so there should be no problem with days that break out
in one
> direction, reverse, and break out in the other; a day that reverses
and ends
> badly will just count as a loss.  Also, I can do the conversion to
intraday
> data myself, rather than ask others to hand me the complete package.
 All I
> need is some way to optimize on a window.
> 
> Offhand, I can't see any way to do it within AFL, and I don't have
the skill
> to handle it with external programming.
> 
> Many thanks.
> 
> Owen Davies
> 
> 
>       Yahoo! Groups Sponsor
>             ADVERTISEMENT
> 
> 
> 
> 
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> 
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.


------------------------ Yahoo! Groups Sponsor ---------------------~-->
Get A Free Psychic Reading! Your Online Answer To Life's Important Questions.
http://us.click.yahoo.com/Lj3uPC/Me7FAA/ySSFAA/GHeqlB/TM
---------------------------------------------------------------------~->

Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/