|
PureBytes Links
Trading Reference Links
|
Jay Mackro wrote:
How can we be confident that
>as a backtest extends further back in time that the characteristics of
>the data are representative of the environment in which the system will
>be trading in real-time?
Good question...I have no immediate answer...maybe Pierre or Bob Brickey
could comment. This is one of my reasons for abandoning EOD systems and
focusing entirely on day trade systems. A one or two year data set (5 tick
basis) is sufficient for my backtesting needs and catches any market
structural changes (like higher volatility) when it occurs.
>Believe me, I very much want to be assured that a system that performed
>well in 1988 will perform well in 1998 - however, does 1988 market data
>really bear any relationship to what I am likely to experience this year?
>I'm not saying it doesn't - I honestly don't know - but it seems plausible
>that the markets are evolving, perhaps at a rate that would invalidate
>backtest results from more than X years ago.
Some systems are trading off different market characteristics that may
appear and disappear and then reappear over time (like cycles). A short
data set for backtesting may not capture these disappearances or
reoccurrences while a large data set would. I guess it depends on what
market characteristic or pattern that you're searching for and how long it
persists backward in time.
>Again, I want to emphasize that I am not debating the desirability of
>consistent system performance over a significant backtest period. The
>legal disclaimers all say that "past results do not indicate future
>performance", but we want to believe that past results are the best
>indicator that we have. Two ideas:
>
> - Might future results be better predicted by backtesting a system
>on recent data from multiple, similar markets, rather than a long
>history from a single market? Ie, look at system results for the last
>two years of T-Bond, T-Bill, and T-Note data, rather than just the last
>six years of just T-Bond data alone. Even, if your plan is to only
>trade the T-Bond market.
>
> - I believe that some guru (Tsuhar Chande?) had proposed a way of
>characterizing market data by statistical parameters - ie. standard
>deviation of daily returns, average range, etc. - and then creating
>"synthetic data" that exhibited similar characteristics. By taking
>the actual characteristics of recent data, and using them to create
>a significant set of synthetic data, extensive backtesting could be
>done against an environment that might better match what would be
>encountered in the immediate future. Anyone accept this approach?
Again, good questions, Jay...
Tony Haas
================================
The commodities market is that creation
of man that humbles him the most.
================================
|