Re: [amibroker] Is the Walk forward study useful?

[Keith McCombs <kmccombs@xxxxxxxxxxxx>]

Gonzaga --
YES, walk forward testing is useful.  Typically, it's purpose is to verify, or not, that your system is robust and not over optimized.  I say 'typically' only because I might imagine that some one might want to try to determine if his system might work better using variable values that worked in the most recent past.  This use would not be 'typical'.

You say, "changing continuously the variables using the walk forward with a 1 month step".  If, you mean that you are using only one month IS for each month OS, your results are telling you that only one month of IS to adjust variables is not sufficient (this is most often the case).  You should probably use a much longer IS, maybe a year or more.

An interesting experiment would be to use 'anchored' IS and compare OS results with with some non-anchored runs.

IS is not a 'test', but training (sorta like being in the students seat next to a flight instructor).  OS is a test (in a flight simulator safely anchored to the ground), which, hopefully, will help prepare you for flying over Manhattan among large flocks of geese.
-- Keith

Gonzaga wrote:

 

I am using the walk forward backtest in order to obtain a realistic CAR data, but I am thinking, since some time ago, that perhaps the walk forward study is not as good as everybody say.

I have a system that gets, during the last 10 years, a 68% annual profit, and a 16% DD.
Of course, I obtain that CAR optimizing one variable, during the last 10 years, and using simultaneous stocks; I mean, is a very very big sample..

The system has few variables, most of them comes from the only indicator I use, with the standard parameters.
Another variable is what I optimize, obtaining the best result in 0.96.
When I perform a walk forward simulation, the CAR goes down to 29%.
Mr Howard Bandy say in his very good book `Quantitative Trading systems' that the in-sample studies are not useful, and we have to consider only out-of-sample studies.
But when I see my in-sample backtest, with a result of 68%, I see that the value of 0.96 has a very good performance every year.
I mean, if I optimize only from 99 to 2003, e.g., the 0.96 value of the variable has a very good performance. And occurs the same in 2003-2007, or in 2007-2009.
In the last two years, that has been so volatile, the 0.96 value is not the best value by far. But, it still has a very good CAR of 105%, although another values has better results.
So I wonder… Is really worth the walk forward? If an optimized variable has a good performance over many years, not always the best, but always a good performance..
Is preferably to use it instead of the madness of changing continuously the variables using the walk forward with a 1 month step???
Thanks, and sorry for my English..

__._,_.___

**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.

TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com

TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___