|
PureBytes Links
Trading Reference Links
|
Gonzaga,
I have similar experience as yours after spending lot of time in backtesting and optimisation. I use WFT for verification of the robustness of the code only. I am going to use the static parameters for real trading since they gave me much better results than WFT.
Once the system stops working, it is time to make a WFT perhaps in order to adjust the parameters.
My static parameters are coming from a splitted data, one is for IS and the other is for OOS. I optimise on IS and backtest on OOS.
I think it is important to find a good fitness factor and also which parameters you WF optimise and in what range. This might be the key for successful WFT.
I would also be interested in reading the experience from other heavy testers.
--- In amibroker@xxxxxxxxxxxxxxx, Keith McCombs <kmccombs@xxx> wrote:
>
> Gonzaga --
> YES, walk forward testing is useful. Typically, it's purpose is to
> verify, or not, that your system is robust and not over optimized. I
> say 'typically' only because I might imagine that some one might want to
> try to determine if his system might work better using variable values
> that worked in the most recent past. This use would not be 'typical'.
>
> You say, "changing continuously the variables using the walk forward
> with a 1 month step". If, you mean that you are using only one month IS
> for each month OS, your results are telling you that only one month of
> IS to adjust variables is not sufficient (this is most often the case).
> You should probably use a much longer IS, maybe a year or more.
>
> An interesting experiment would be to use 'anchored' IS and compare OS
> results with with some non-anchored runs.
>
> IS is not a 'test', but training (sorta like being in the students seat
> next to a flight instructor). OS is a test (in a flight simulator
> safely anchored to the ground), which, hopefully, will help prepare you
> for flying over Manhattan among large flocks of geese.
> -- Keith
>
> Gonzaga wrote:
> >
> >
> > I am using the walk forward backtest in order to obtain a realistic
> > CAR data, but I am thinking, since some time ago, that perhaps the
> > walk forward study is not as good as everybody say.
> >
> > I have a system that gets, during the last 10 years, a 68% annual
> > profit, and a 16% DD.
> > Of course, I obtain that CAR optimizing one variable, during the last
> > 10 years, and using simultaneous stocks; I mean, is a very very big
> > sample..
> >
> > The system has few variables, most of them comes from the only
> > indicator I use, with the standard parameters.
> > Another variable is what I optimize, obtaining the best result in 0.96.
> > When I perform a walk forward simulation, the CAR goes down to 29%.
> > Mr Howard Bandy say in his very good book `Quantitative Trading
> > systems' that the in-sample studies are not useful, and we have to
> > consider only out-of-sample studies.
> > But when I see my in-sample backtest, with a result of 68%, I see that
> > the value of 0.96 has a very good performance every year.
> > I mean, if I optimize only from 99 to 2003, e.g., the 0.96 value of
> > the variable has a very good performance. And occurs the same in
> > 2003-2007, or in 2007-2009.
> > In the last two years, that has been so volatile, the 0.96 value is
> > not the best value by far. But, it still has a very good CAR of 105%,
> > although another values has better results.
> > So I wonder... Is really worth the walk forward? If an optimized
> > variable has a good performance over many years, not always the best,
> > but always a good performance..
> > Is preferably to use it instead of the madness of changing
> > continuously the variables using the walk forward with a 1 month step???
> > Thanks, and sorry for my English..
> >
> >
>
------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|