|
PureBytes Links
Trading Reference Links
|
Tim,
The sad fact is that the ONLY WAY to properly test a canned "black-box"
system is to test it on new data that the system has never seen. Since you
have no way of knowing what past data the system has seen and hasn't seen,
the only option is to test the system on future data, i.e., paper trade the
system (or, heaven forbid, trade it with real money).
Unfortunately, unless the system you buy is an active day-trading system
(most trading systems you can buy are not - they are position trading
systems), it will take you a LONG TIME to build up a statistically
significant number of trades to see how the system performs. Perhaps 100
trades, certainly no less than 30.
Example: You buy a system for trading bonds. Without being able to see the
code, how do you know that they system has not been optimized with simple
date-based parameters that make it seem very successful on historical data?
(An example of this practice was recently posted to the list.) The only
solution is to test the system on data that it could not possibly have seen.
Because of the difficulty outlined above, the purchasing of "locked" systems
is a risky proposition at best. This is not to say that all black-box
systems are rip-offs. Only that it is very difficult to tell which ones are
and which ones are not.
Caveat: My understanding is that the Futures Truth organization does take
some steps to test black-box systems that are sold to the public. I would
contact them for more details of their procedures and results before I
bought any system that did not disclose its underlying logic to me. Perhaps
someone else knows more details about this.
Blaine Mathieu
Turning Point Trading Inc.
>Ok, so I get the little file. And now that I've read the post about
>the vendors masking the true system performance, I'm a very nervous
>consumer. So I want to know how to test this new toy. How do you test
>a system, say a break-out system or something, that is used on a
>portfolio of commodities and then trades on the given active months in
>those markets? I couldn't very well trust the vendor's word after
>reading what I've read here.
|