|
PureBytes Links
Trading Reference Links
|
Hello,
Another good book concerning the topic was recommended by S. Karnish
some months ago.
'Smart Momentum : The Future of Predictive Analysis in the Financial
Markets' by Hugh Clark
Regards
Leo
--- In amibroker@xxxxxxxxxxxxxxx, "Herman van den Bergen"
<psytek@xxxx> wrote:
> Yes ken/bill, I did misunderstand the procedure :-( actually all
test
> windows are OOS. Thanks for explaining things for me.
>
> I wonder how they would determine the Optimize/Test period
ratio... the
> example uses 2:1. A critical factor that is not mentioned is trade
duration.
> It would seem prudent to have a reasonable number of trades in
both the and
> test periods. So this system would only work with short term
trading
> systems...if you trades last more then 10% of the test period you
wouldn't
> really have enough data to make results significant.
>
> With spread of optimization values i mean the range of the
parameters
> optimized. Suppose the system uses period optimization and the
values, over
> 18 tests, would vary from 3 to 12... I would be highly skeptical
this type
> of result.
>
> Like was mentioned: there is no easy way...
>
> herman
> -----Original Message-----
> From: Ken Close [mailto:closeks@x...]
> Sent: Saturday, March 01, 2003 3:38 PM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: RE: [amibroker] Re: Automated Backtesting Walkforward
Validation
>
>
> Herman:
>
>
>
> I would tend to think that it does tell you something about the
system
> under test.
>
>
>
> If your results in the "Trade Window" as it is called in this
message are
> good, and they are "good" in say 15 out of 18 periods or cycles,
then is it
> not reasonable to assume that the system under test is "robust"
and is
> likely to perform ok (or at least as good as the average
performance
> achieved in all of the Trade Windows).
>
>
>
> Yes, I interpreted the msg to say to reoptimize in each
new "Test Window",
> the walk forward test in each Trade Window. I am not sure what
would happen
> if the beginning date of the first Test Window was anchored and
the Test
> Window got progressively longer after each cycle. It is also not
clear what
> minimum number of trades must be achieved for the Test Window and
Trade
> Window to be of adequate length.
>
>
>
> What did you mean by "spread of optimization values"???
>
>
>
> Ken
>
>
>
> -----Original Message-----
> From: Herman vandenBergen [mailto:psytek@x...]
> Sent: Saturday, March 01, 2003 9:14 PM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: RE: [amibroker] Re: Automated Backtesting Walkforward
Validation
>
>
>
> The previous post read: "Start with a fraction of the total
historical
> data, say 10%, call this the Test-Window, run the Optimizer on
this data,
> apply the
> trading system with these optimized parameters to the period of
data
> immediately following the Test-Window, say 5% of total historical
> data, call this the Trade-Window, record only the Trade-Window
> results, move the starting point of the Test-Window ahead by the
> length of the Trade-Window, repeat (Does this mean re-
optimize?), in this
> case 18 times, until the walk-forward process reaches the end of
the
> historical data, and then report the cumulative Trade-Window
results."
>
>
>
> If i interpret this correctly than this really is an assessment
of the
> optimization process itself. If so it has liitle to do with the
trading
> system under test and has no value in predicting the effectivenes
of the
> last (all previous values have no relevance) optimization values
in the
> trading system on the next future period. There is not even talk
about
> measuring the spread of optimization values...
>
>
>
> Hopefully I misunderstood...
>
>
>
> Herman.
>
>
>
> -----Original Message-----
> From: bvandyke <bvandyke@xxxx> [mailto:bvandyke@x...]
> Sent: March 1, 2003 11:48 AM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: [amibroker] Re: Automated Backtesting Walkforward
Validation
>
> Hi d,
>
> The best definition of Walkforward Validation (WFV) i saw is as
> follows and from a program called Inference Trader. A google
search
> will show their website.
>
> "WalkforwardValidation (WFV) is a method of back-testing a
trading
> system using walk-forward optimization/trading. It attempts to
> simulate a more realistic trading system performance than the
> Optimization only--whose trading results are overly
optimistic. The
> method is as follows:
>
> Start with a fraction of the total historical data, say 10%,
call
> this the Test-Window, run the Optimizer on this data, apply the
> trading system with these optimized parameters to the period
of data
> immediately following the Test-Window, say 5% of total
historical
> data, call this the Trade-Window, record only the Trade-Window
> results, move the starting point of the Test-Window ahead by
the
> length of the Trade-Window, repeat, in this case 18 times,
until
> the walk-forward process reaches the end of the historical
data, and
> then report the cumulative Trade-Window results."
>
> "If the results as described above are positive,then one
usually to
> get the parameters to use for future trading, uses the
Optimizer.
> Since you back-tested this strategy of optimize/trade with
> WFV optimizing the paramaters on a chunk of data "Test window"
length
> long, to attempt to recreate this performance in future
> trading, you want to set the optimization length equal to "Test
> window" length. Now run the optimizer on your last "test
window"
> period of data to get the parameters for future trading. Now
trade
> for a period of time equal to the WFV "Trade Window Length",
at the
> end of this period, run the optimizer again then trade with
the new
> params, and so on."
>
> Hope this helps,
> Bill
>
> --- In amibroker@xxxxxxxxxxxxxxx, "dingo" <dingo@xxxx> wrote:
> > can you point me to some more info on "Walkforward
Optimization"??
> >
> > d
> >
> > -----Original Message-----
> > From: bvandyke <bvandyke@xxxx> [mailto:bvandyke@x...]
> > Sent: Saturday, March 01, 2003 1:35 PM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: [amibroker] Re: Automated Backtesting
> >
> >
> > Hi dingo,
> >
> > You wrote below about Optimization:
> > >I should point out that according to most of the
knowledgeable
> > >sources that you should only optimize once for a range of
dates
> that
> > will include bull and bear periods and then do a series of
> > backtests "out of sample" to find out if your formula
is "robust".
> > That's the hard part!
> > >
> > > d
> >
> > I think Leo may be referring, in his Post, to an
Optimization
> > technique called "Walkforward Validation", which is what some
> people
> > recommend and prefer. It's another option to handle bull
and bear
> > years. As you say, no matter what technique, it's not
easy :)
> >
> > Bill
> >
> >
> >
> > >
> > > -----Original Message-----
> > > From: leo_amelc <leo.timmermans.lt@xxxx>
> > > [mailto:leo.timmermans.lt@x...]
> > > Sent: Saturday, March 01, 2003 10:09 AM
> > > To: amibroker@xxxxxxxxxxxxxxx
> > > Subject: [amibroker] Automated Backtesting
> > >
> > >
> > > Hello,
> > >
> > > I've a question regarding the AA Automation objects.
> > >
> > > I want to do the following:
> > >
> > > 1) take a stock (or group of stocks) and optimize an
indicator
> > > during a period ((RangeFromDate - RangeToDate) of let's
say 6
> > months.
> > >
> > > 2) use the 'best values' to backtest this stock during a
certain
> > > period; this period directly follows the optimisation
period
> > >
> > > 3) repeat steps 1 and 2 in a 'rolling mode'.
> > >
> > > My question is : how can I get the optimised values in my
> > backtest ??
> > > I guess one way is through the export of the results list
to CSV
> > file
> > > but maybe there is a smarter way ??
> > >
> > > Thanks
> > > Leo
> > >
> > >
> > >
> > >
> > > Yahoo! Groups Sponsor
> > >
> > > ADVERTISEMENT
> > >
> > >
> >
>
<http://rd.yahoo.com/M=243376.3035052.4359433.1927555/D=egroupweb/S=1
7
> > 05
> > >
> >
>
632198:HM/A=1414910/R=0/*http://ad.doubleclick.net/jump/N879.ameritra
d
> > e.
> > > yahoo/B1054521.29;sz=300x250;adc=zhs;ord=1046531329358607?>
> > >
> > > <http://us.adserver.yahoo.com/l?
> > M=243376.3035052.4359433.1927555/D=egrou
> > > pmail/S=:HM/A=1414910/rand=487282882>
> > >
> > > Send BUG REPORTS to bugs@xxxx
> > > Send SUGGESTIONS to suggest@xxxx
> > > -----------------------------------------
> > > Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx
> > > (Web page:
http://groups.yahoo.com/group/amiquote/messages/)
> > > --------------------------------------------
> > > Check group FAQ at:
> > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > >
> > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> Service
> > > <http://docs.yahoo.com/info/terms/> .
> >
> >
> >
> > Yahoo! Groups Sponsor
> >
> > ADVERTISEMENT
> >
> >
>
<http://rd.yahoo.com/M=243376.3035052.4359433.1927555/D=egroupweb/S=1
7
> 05
> >
>
632198:HM/A=1414910/R=0/*http://ad.doubleclick.net/jump/N879.ameritra
d
> e.
> > yahoo/B1054521.29;sz=300x250;adc=zhs;ord=1046543713844329?>
> >
> > <http://us.adserver.yahoo.com/l?
> M=243376.3035052.4359433.1927555/D=egrou
> > pmail/S=:HM/A=1414910/rand=941839877>
> >
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
Service
> > <http://docs.yahoo.com/info/terms/> .
>
>
>
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
>
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of
Service.
>
>
>
>
>
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
>
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of
Service.
>
>
>
>
> Yahoo! Groups Sponsor
> ADVERTISEMENT
>
>
>
>
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
>
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of
Service.
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Rent DVDs Online-No late fees! Try Netflix for FREE!
http://us.click.yahoo.com/bbvVKB/oEZFAA/46VHAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|