|
PureBytes Links
Trading Reference Links
|
Yes
ken/bill, I did misunderstand the procedure :-( actually all test
windows are OOS. Thanks for explaining things for me.
<FONT face=Arial color=#0000ff
size=2>
I
wonder how they would determine the Optimize/Test period ratio... the example
uses 2:1. A critical factor that is not mentioned is trade duration. It would
seem prudent to have a reasonable number of trades in both the and test periods.
So this system would only work with short term trading systems...if you trades
last more then 10% of the test period you wouldn't really have enough data to
make results significant.
<FONT face=Arial color=#0000ff
size=2>
With
spread of optimization values i mean the range of the parameters optimized.
Suppose the system uses period optimization and the values, over 18 tests, would
vary from 3 to 12... I would be highly skeptical this type of
result.
<FONT face=Arial color=#0000ff
size=2>
Like
was mentioned: there is no easy way...
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>herman
<FONT face=Tahoma
size=2>-----Original Message-----From: Ken Close
[mailto:closeks@xxxxxxxx]Sent: Saturday, March 01, 2003 3:38
PMTo: amibroker@xxxxxxxxxxxxxxxSubject: RE: [amibroker]
Re: Automated Backtesting Walkforward Validation
<SPAN
>Herman:
<SPAN
>
<SPAN
>I would tend to think
that it does tell you something about the system under test.
<SPAN
>
<SPAN
>If your results in
the “Trade Window” as it is called in this message are good, and they are
“good” in say 15 out of 18 periods or cycles, then is it not reasonable to
assume that the system under test is “robust” and is likely to perform ok (or
at least as good as the average performance achieved in all of the Trade
Windows).
<SPAN
>
<SPAN
>Yes, I interpreted
the msg to say to reoptimize in each new “Test Window”, the walk forward test
in each Trade Window. I am not sure what would happen if the beginning
date of the first Test Window was anchored and the Test Window got
progressively longer after each cycle. It is also not clear what minimum
number of trades must be achieved for the Test Window and Trade Window to be
of adequate length.
<SPAN
>
<SPAN
>What did you mean by
“spread of optimization values”???
<SPAN
>
<SPAN
>Ken
<SPAN
>
<SPAN
>-----Original
Message-----From: Herman
vandenBergen [mailto:psytek@xxxxxxxx] <SPAN
>Sent: Saturday, March 01, 2003 9:14
PMTo:
amibroker@xxxxxxxxxxxxxxx<SPAN
>Subject: RE: [amibroker] Re: Automated
Backtesting Walkforward Validation
<SPAN
>
<SPAN
>The previous post
read: "<SPAN
>Start with a fraction of the
total historical data, say 10%, call this the Test-Window, run the Optimizer
on this data, apply the trading system with these optimized parameters to
the period of data immediately following the Test-Window, say 5% of total
historical data, call this the Trade-Window, record only the Trade-Window
results, move the starting point of the Test-Window ahead by the
length of the Trade-Window, repeat <FONT face="Courier New"
color=blue>(Does this
mean re-optimize?),<SPAN
> in this case 18 times, until
the walk-forward process reaches the end of the historical data, and then
report the cumulative Trade-Window results."
<SPAN
>
<SPAN
>If i
interpret this correctly than this really is an assessment of the optimization
process itself. If so it has liitle to do with the trading system under test
and has no value in predicting the effectivenes of the last (all previous
values have no relevance) optimization values in the trading system on the
next future period. There is not even talk about measuring the spread of
optimization values...
<SPAN
>
<SPAN
>Hopefully I
misunderstood...
<SPAN
>
<SPAN
>Herman.
<SPAN
>
<FONT face=Tahoma
size=2>-----Original
Message-----From: bvandyke
<bvandyke@xxxxxxxxxxxxx> [mailto:bvandyke@xxxxxxxxxxxxx]<SPAN
>Sent: March 1, 2003 11:48
AMTo:
amibroker@xxxxxxxxxxxxxxx<SPAN
>Subject: [amibroker] Re: Automated
Backtesting Walkforward Validation
<SPAN
>Hi d,<FONT face="Courier New"
size=2><SPAN
><FONT
face="Courier New">The best definition of Walkforward Validation (WFV) i saw
is as follows and from a
program called Inference Trader. A google search
will show their
website.<FONT
face="Courier New">"WalkforwardValidation (WFV) is a method of back-testing
a trading system using
walk-forward optimization/trading. It attempts to
simulate a more realistic
trading system performance than the <FONT
face="Courier New">Optimization only--whose trading results are overly
optimistic. The method is
as follows:Start with a
fraction of the total historical data, say 10%, call
this the Test-Window, run the
Optimizer on this data, apply the <FONT
face="Courier New">trading system with these optimized parameters to the
period of data immediately
following the Test-Window, say 5% of total historical
data, call this the
Trade-Window, record only the Trade-Window <FONT
face="Courier New">results, move the starting point of the Test-Window ahead
by the length of the
Trade-Window, repeat, in this case 18 times, until<FONT
face="Courier New">the walk-forward process reaches the end of the
historical data, and then
report the cumulative Trade-Window results."<FONT
face="Courier New">"If the results as described above are positive,then one
usually to get the parameters
to use for future trading, uses the Optimizer.
Since you back-tested this
strategy of optimize/trade with<FONT
face="Courier New">WFV optimizing the paramaters on a chunk of data "Test
window" length long, to attempt
to recreate this performance in future<FONT
face="Courier New">trading, you want to set the optimization length equal to
"Test window" length. Now run
the optimizer on your last "test window" <FONT
face="Courier New">period of data to get the parameters for future
trading. Now trade for a
period of time equal to the WFV "Trade Window Length", at the
end of this period, run the
optimizer again then trade with the new <FONT
face="Courier New">params, and so on."<FONT
face="Courier New">Hope this helps,<FONT
face="Courier New">Bill
--- In
amibroker@xxxxxxxxxxxxxxx, "dingo" <dingo@xxxx>
wrote:> can you point me to
some more info on "Walkforward Optimization"??
>
> d<FONT
face="Courier New">> >
-----Original Message----->
From: bvandyke <bvandyke@xxxx> [mailto:bvandyke@xxxx]
> Sent: Saturday, March 01,
2003 1:35 PM> To:
amibroker@xxxxxxxxxxxxxxx>
Subject: [amibroker] Re: Automated Backtesting<FONT
face="Courier New">> >
> Hi
dingo,>
> You wrote below about
Optimization:> >I should
point out that according to most of the knowledgeable
> >sources that you
should only optimize once for a range of dates <FONT
face="Courier New">that >
will include bull and bear periods and then do a series of
> backtests "out of sample"
to find out if your formula is "robust". <FONT
face="Courier New">> That's the hard part!<FONT
face="Courier New">> > <FONT
face="Courier New">> > d<FONT
face="Courier New">> > I
think Leo may be referring, in his Post, to an Optimization
> technique called
"Walkforward Validation", which is what some <FONT
face="Courier New">people >
recommend and prefer. It's another option to handle bull and bear
> years. As you say,
no matter what technique, it's not easy :)<FONT
face="Courier New">> >
Bill>
> <FONT
face="Courier New">> >
> > > -----Original
Message-----> > From:
leo_amelc <leo.timmermans.lt@xxxx><FONT
face="Courier New">> > [mailto:leo.timmermans.lt@xxxx]
> > Sent: Saturday, March
01, 2003 10:09 AM> > To:
amibroker@xxxxxxxxxxxxxxx>
> Subject: [amibroker] Automated Backtesting<FONT
face="Courier New">> > <FONT
face="Courier New">> > <FONT
face="Courier New">> > Hello,<FONT
face="Courier New">> > <FONT
face="Courier New">> > I've a question regarding the AA Automation
objects.> >
> > I want to do the
following:> >
> > 1) take a stock (or
group of stocks) and optimize an indicator <FONT
face="Courier New">> > during a period ((RangeFromDate - RangeToDate)
of let's say 6 >
months.> >
> > 2) use the 'best
values' to backtest this stock during a certain <FONT
face="Courier New">> > period; this period directly follows the
optimisation period> >
> > 3) repeat steps 1 and
2 in a 'rolling mode'.> >
> > My question is : how
can I get the optimised values in my <FONT
face="Courier New">> backtest ??<FONT
face="Courier New">> > I guess one way is through the export of the
results list to CSV >
file> > but maybe there
is a smarter way ??> >
> >
Thanks> >
Leo> >
> >
> >
> >
> > Yahoo! Groups
Sponsor <FONT
face="Courier New">> > <FONT
face="Courier New">> > ADVERTISEMENT<FONT
face="Courier New">> > <FONT
face="Courier New">> > <FONT
face="Courier New">> <<A
href="">http://rd.yahoo.com/M=243376.3035052.4359433.1927555/D=egroupweb/S=17<FONT
face="Courier New">> 05>
> >
632198:HM/A=1414910/R=0/*<A
href="">http://ad.doubleclick.net/jump/N879.ameritrad<FONT
face="Courier New">> e.>
>
yahoo/B1054521.29;sz=300x250;adc=zhs;ord=1046531329358607?>
> >
> > <<A
href="">http://us.adserver.yahoo.com/l?<FONT
face="Courier New">>
M=243376.3035052.4359433.1927555/D=egrou<FONT
face="Courier New">> >
pmail/S=:HM/A=1414910/rand=487282882>
> >
> > Send BUG REPORTS to
bugs@xxxx> > Send
SUGGESTIONS to suggest@xxxx>
> -----------------------------------------<FONT
face="Courier New">> > Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx >
> (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)<FONT
face="Courier New">> >
--------------------------------------------<FONT
face="Courier New">> > Check group FAQ at:<FONT
face="Courier New">> > <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> > Your use of Yahoo!
Groups is subject to the Yahoo! Terms of <FONT
face="Courier New">Service>
> <<A
href="">http://docs.yahoo.com/info/terms/>
.> <FONT
face="Courier New">> >
> Yahoo! Groups
Sponsor <FONT
face="Courier New">> >
ADVERTISEMENT>
> <FONT
face="Courier New"><<A
href="">http://rd.yahoo.com/M=243376.3035052.4359433.1927555/D=egroupweb/S=17<FONT
face="Courier New">05>
632198:HM/A=1414910/R=0/*<A
href="">http://ad.doubleclick.net/jump/N879.ameritrad<FONT
face="Courier New">e.>
yahoo/B1054521.29;sz=300x250;adc=zhs;ord=1046543713844329?>
<FONT
face="Courier New">> <FONT
face="Courier New">> <<A
href="">http://us.adserver.yahoo.com/l?<FONT
face="Courier New">M=243376.3035052.4359433.1927555/D=egrou<FONT
face="Courier New">> pmail/S=:HM/A=1414910/rand=941839877>
<FONT
face="Courier New">> >
Send BUG REPORTS to bugs@xxxx<FONT
face="Courier New">> Send SUGGESTIONS to
suggest@xxxx>
-----------------------------------------<FONT
face="Courier New">> Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx >
(Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)<FONT
face="Courier New">>
--------------------------------------------<FONT
face="Courier New">> Check group FAQ at:<FONT
face="Courier New">> <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> <FONT
face="Courier New">> Your use of Yahoo! Groups is subject to the Yahoo!
Terms of Service> <<A
href="">http://docs.yahoo.com/info/terms/>
.<FONT face="Courier New"
size=2>Send BUG REPORTS to
bugs@xxxxxxxxxxxxx<SPAN
><FONT
face="Courier New">Send SUGGESTIONS to
suggest@xxxxxxxxxxxxx<FONT
face="Courier New">-----------------------------------------<FONT
face="Courier New">Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx (Web
page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)<FONT
face="Courier New">--------------------------------------------<FONT
face="Courier New">Check group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
<SPAN
>Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of
Service.
<SPAN
><FONT
face="Courier New" size=2>Send BUG REPORTS to
bugs@xxxxxxxxxxxxx<SPAN
><FONT
face="Courier New">Send SUGGESTIONS to
suggest@xxxxxxxxxxxxx<FONT
face="Courier New">-----------------------------------------<FONT
face="Courier New">Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx (Web
page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)<FONT
face="Courier New">--------------------------------------------<FONT
face="Courier New">Check group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your
use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of
Service.
Send
BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Yahoo! Groups Sponsor
ADVERTISEMENT
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
|