|
PureBytes Links
Trading Reference Links
|
TH wrote:
>> What my rules / guidelines / suggestions try to do is
>> simply weed out the dishonest or even sloppy system vendors
>> by requiring documented real time performance results...<SNIP>
BB wrote:
>Good real-time results tend to give us all a warm fuzzy feeling. However,
>contrary to the belief of most traders, real-time trading results often mean
>very little. They don't mean much, unless there have been enough trades to
>have statistical significance.
<snip>
The problem with most system sellers is the lack of rigorous backtesting.
A lot of vendors only use 5 years EOD and 1 year daytrade data for system
backtesting, hardly adequate for any statistical validation. I agree with
you and Pierre when you both demand large data sets for system backtesting.
But I disagree with your premise that real time performance results lack
statistical relevance.
One of my points in an earlier post is that I require a minimum of 3 years
real time account performance for EOD systems and 1 year for daytrade
systems. Why did I pick these numbers? Because a lot of system vendors
use only 5 and 1 years, respectively, for backtesting EOD and daytrade
systems. These systems are typically over-optimized on a data set favorable
to the system parameters. If the system is going to eventually
self-destruct, then I should see a divergence between the backtest results
and the real time account performance. It does and I have, many times.
Better for the system to self-destruct in the vendors backyard than in my
trading account.
Tony Haas
================================
The commodities market is that creation
of man that humbles him the most.
================================
|