|
PureBytes Links
Trading Reference Links
|
Dans un courrier daté du 25/02/98 21:16:03 , Tony Haas écrit :
<<
>If you backtest on such amount of data, with realistic slippage, this is as
>good as an audited account (at least for the developer that may rely on his
>probity).
Extremely important...I fully agree. I believe that a lot of mechanical
systems fail over time because the developer was sloppy in his / her work.
A system is as sound as the foundation upon which it is built, and that
foundation is good clean data and a lots of it.
>>
All of this is crucial.
I can only speak for myself.
¨Most of the testing is wade on unseen data, and a such amount that it is
impossible
to have a wrong answer on the real performance of the system.
I usually train on some thousand bar and test on 50-100 times more .
>From my poinbt of view, if the system is to be traded and if I'm paid for
that, I have noo reason
to fool myself and the final user.
<<
I have a minor disagreement over the statement that a good backtest (with
realistic slippage) is as good as an audited account. While I use a those
standards in my own system development, I require a higher standard from
those who sell systems. Why? When I test my ideas, I'm all too familar
with them (sometimes familarity breeds contempt <g>). But if I buy a black
or gray box system, I'm not privy to the system rules and math, just the
output...buy, sell, or flat. And as we all know too well, systems can be
tweaked to provide a realistic performance in the backtest, but fall apart
in real time trading.
>>
The problem occurs if you sell the system.
No guarantee that the backtest was not crooked.
The answer is no in my case, as I'pm the developer, the tester and a part of
the
final user. Others cases may of course differ.
<<
I can anticipate your next question: But Pierre uses backtest data set and
reserves a portion of the data for real time performance evaluation! I
have to agree that this approach is better than none at all, but a lot of
system vendors are not as thorough, ie, more slopp
higher standard thing again. Real time account
performance is measured by executing actual trades and testing the system
in the trenches under market conditions. There's really no better
substitute than actual market combat conditions.
>>
Sure, it's the only objective test.
But if it fails , then you have the proof that the system was not so good
and you have lst money.
I prefer a huge backtest (but a very huge one) to a realtime test.
It takes too much time, and is more risky in my (developer/ partial user)
opinion.
When I'll have time , I'll try to post here some crazy overoptimized system.
You will see that I know something about what I speak about backtesting
and its intrinsic limits knowledge.
Sincerely,
Pierre Orphelin
www.sirtrade.com
|