|
PureBytes Links
Trading Reference Links
|
Hi Howard,
Thank you for your reply. I must say that your book - very interesting, BTW - is another reason why I am skeptical with the trend following strategy based on a single "good-ever" code.
1) But the problem I have with limited data is how they react to today's market. As an example, I built a system which returns avg. 100% for the last 4 years. But considering that 3 and a half years were a strong bull market, how can I be sure this apply to today's condition? On the other side, if I take only data from November 2007 to April 2008, this is not enough to be sure profits are not a consequence of the "luck" factor.
This is why I would have enjoyed having data from the other bear market, from 2000 to 2003. And this is why I am not sure if building a system over the past 6months to 12 months would be a good idea.
2) Another problem I have is that part of the system is based on a major index. Even if I did a lot of optimization (over 1000) and obtained good results, my fear is that the whole thing is biased because it could be a data-mining bias as the particular index in concerned. In other words: if I take only 6 months of data and my system issues only 3 buying signals based in a big part on only one major index, then I think statistically this is NOT significant. This is why I tried to look for much longer time-frame (back to 2000, with the end of the techno bubble), but then I am back to the initial problem: did the market change?
To me, this is a serious problem. I could feel happy because my system seems good but I am not. I don't think we are in a market right now that could even be remotely related to what happened in 2006 and 2007.
Thanks!
Louis
2008/4/8, Howard B <howardbandy@xxxxxxxxx>:
Hi Louis, and all --
I know David Aronson, respect him, and like and recommend his book.
My view is that the in-sample period should be as short as practical. My thought is that: the system we are testing / trading is trying to recognize the signal from among the noise; and the signal patterns are changing over time. So the length of the in-sample period is a tradeoff -- short to be able to change as the characteristics of the underlying market change, but not so short that the system is over-fit to the noise rather than learns the signal.
You can test this in AmiBroker. Have your system ready to buy and sell. In the Automatic Analysis window, use Settings and set up the Walk Forward parameters. Try an in-sample period of 10 years, an out-of-sample period of 6 months or 1 year. Run Optimize > Walk Forward and look at the in-sample and out-of-sample equity curves. Shorten the length of the in-sample period to 9, then 8, then 7, ... then 1 year, keeping the out-of-sample period unchanged. Depending on your system and the market it is trading, you may find that there is a sweet spot in the length of the in-sample data. If so, that is the amount of data that allows your system to recognize the signal without being overwhelmed by the noise.
Thanks,
Howard
On Tue, Apr 8, 2008 at 8:56 AM, Louis Préfontaine < rockprog80@xxxxxxxxx> wrote:
Hi,
I've been thinking a lot lately, and here is something I would like to have your opinion on.
I've been introduced to automated systems by a trend following book which related how some trend followers built their systems in the 70s or 80s and got rich with them, and how their system did not really change all this time. They didn't change their system because they say the market does NOT change. They looked at historic market data from the 1800s and the market was as it is right now. So they say.
On the other side, lately I have been introduced to the concept of ever-changing markets and have had a hard time trying to build my system. Got a very promising start with a system getting around 15-20% average for April 2007 to April 2008 (with little drawdown, which mean that with leverage I can boost this a lot). In any variation over thousands of stocks the results were nearly all positives. But then, I tested that same system for the years 2000 to 2008, and that was disappointing. Even more disappointing from 2001 to 2003, another troubled market like the one we are in right now.
So here I am, wondering where to go from now. Aronson's excellent book talk about the importance of having a very large sample of data. But the problem is: the larger the data, the more "historic" it gets and the less it seems to work.
Is my system not working, or did the markets really change? Do I need to make it more robust (that is, it MUST make profit even from 2001 to 2003), or can I have complete faith in what happened in the last year?
All those questions... Would be nice to read what you think about this.
Louis
__._,_.___
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
![]()
__,_._,___
|