|
PureBytes Links
Trading Reference Links
|
Hi Chris --
You can do anything you want to in your search for a good trading system. The data period you work with during that search is the in-sample period. The results you achieve over the in-sample period have no value in predicting what the future performance will be. In order to estimate the future performance, you need to test the program on a set of data that follows the in-sample period and has not been used at all in the development of the system. That data is called the out-of-sample data. You can perform statistical tests on the out-of-sample results, but the quickest way to evaluate it is to look at the out-of-sample equity curve.
Be careful to avoid the following procedure. Optimize in-sample, evaluate out-of-sample, modify the system based on the the out-of-sample results, retest out-of-sample. The previously out-of-sample data period has become part of an expanded in-sample data set and a new out-of-sample test is required in order to estimate future performance.
There is a lot more to system development, testing, and validation than those two paragraphs. I am presenting a two-day workshop in Las Vegas February 21 and 22 devoted to that subject.
http://www.ftmonitor.com/lv08/lv08intro.html
And I have written a book devoted to that subject.
http://www.quantitativetradingsystems.com/
Thanks,
Howard
On Feb 5, 2008 4:34 AM, ChrisB < kris45mar@xxxxxxxxxxxx> wrote:
What is a valid or reasonable number of backtest results to subject to
Optimization?
For general statistics a minimum of 30 or so is needed to start getting
valid StdDevs etc.
If I run a backtest on hourly currency data over three months I get
around 16 -20 tradeable signals per currency.
This give a nice smooth plateau on 3D optimization.
If I test over two months of data I get around 10 - 12 trades
If I test over only 1 month I get only 5 or 6 trades.
These shorter time periods still give visually acceptable 3D plateaus
but I am wondering if there is enough data to be statistically significant.
I am trying to get a handle on how close I can get to current
fluctuations in the market without hitting noise. The idea being to redo
the Optimization every x time frame and shift the entry and exit
parameters to stay in the middle of the plateau.
Of course I can backtest over longer time frames, say 6 months of data,
shifting the starting date forward by one month at a time, but this
would seem to introduce more "lag" into my selection of best parameters
to trade.
Does anyone have any thoughts/references on this?
--
Regards
ChrisB
__._,_.___
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
![]()
__,_._,___
|