Re: [amibroker] Optimizing, AmiBroker Email List Archive

Any takers on this one? For instance I was just testing a system on a certain intraday interval. I then tested it against the same interval but in increments of +1 and -1, from -5 to +5. For the most part it was stable, in the sense that the more I incremented it the better it did, and the more I decremented it the worse it did, but never falling below a respectable Sharpe or % return, although at the low end it was 50% of the total return of the in sample version. This tells me that my signal is mis-timed to the market, but it seems to be mistimed in a consistent way. Thoughts?

From: Potato Soup <potatosoupz@xxxxxxxxx>
To: AmiBroker (Discussion List) <amibroker@xxxxxxxxxxxxxxx>
Sent: Wed, January 6, 2010 5:16:48 PM
Subject: Re: [amibroker] Optimizing

In sample is always bad to rely on. But how do you define out of sample? If you are backtesting one symbol with intraday data and less than 100 trades per year, do you start with the oldest year and optimize, then test out of sample the newer years? Or do you start with the best perfoemong years, hoping that it improves the worst performing years? Or do you test all years and then verify by using another symbol as out of sample?

From: Howard B <howardbandy@xxxxxxxxx>

Date: Wed, 6 Jan 2010 14:30:45 -0700

To: <amibroker@xxxxxxxxxxxxxxx>

Subject: Re: [amibroker] Optimizing

Hi Markus --

The characteristics of a desirable trading system are yours to decide. Whether you want to focus on trend following systems, on mean reversion systems, on pattern systems, statistical systems, or whatever else is completely up to you.

I meant no criticism. My suggestion about allowing the relationship between the two moving average lengths was simply to point out that what was originally thought of as a trend following system might transform itself into a mean reversion system under some circumstances.

As always -- do your own research, including in-sample testing and out-of-sample validation. Walk forward testing is extremely valuable. In-sample results are always good and have no value in estimating future performance of a system.

Thanks,
Howard