|
PureBytes Links
Trading Reference Links
|
I have a question I've rarely seen addressed. M. Edward you seem to have
alot of systems and stats knowledge, perhaps you can help with this one.
What are some good ways to pragmatically decide that a system is no longer
performing at acceptable levels?
I guess one answer is when the aggregate results (history plus post
development "real-time" results) are below the minimum thresholds personally
set for trading a system. But a system that suddenly goes belly up could
eat up alot of equity before that point is reached.
Ralph Vince suggested basing a decision to terminate trading on the largest
historical intraday drawdown being exceeded, and not resuming trading until
new equity highs are reached (would have been reached if trading wasn't
terminated!). This strikes me as somewhat conservative in the same way
his compounding based on largest historical loss is too aggressive; the
"largest" is always in the future and eventually we'll see a new largest.
Another approach would be to use probabilities of M losses in N trades,
and anytime the actual occurance based on historical percentages had a
less than X% (i.e., 1%) chance of occuring, time to call it quits, the
system is giving strong indications of failure.
I'm loath to make decisions about system termination based on average trade
results from non-significant sample sizes (i.e., I've got ten actual trades
and the average return per trade is $250 while the historical average has
been $600) as a basis for bailing.
I am actively trading 3 mechanical systems, so this is very much a working
question for me. My approach right now is "commit and execute for six months
and then reevaluate", which I feel is necessary from a discipline perspective
(I've only broken down once so far and made a discretionary trade, got out
after a day and am re-committed to not doing that!). However, it runs the
risk of losing money unnecessarily if the results are giving strong
statistical indications of breakdown.
My experience is that the markets will test our resolve in incredible ways.
I watched my three systems run extremely well in real-time after initial
development, for six months, not trading them (I watched many others I
developed and found them to be crap or weak, both in real-time and with
more analysis on more data, while these held up in all ways). I started
trading my three best in December. I suffered a sequence of losses in all of
them, each of which alone was within the bounds of reasonable system behavior
(no new highs set of sequence of losses or drawdown), BUT when viewed together,
the independent odds of all of them firing that way (5 losses across the three
of them "in a row") at the same time was down in the 0.02% range. I was and
continue to be amazed, and yet, this can happen, did happen, and by no
means proves these systems are no longer functioning within bounds.
(Since then I've got two winners, with a third winner very close to exiting.)
Any suggestions on my question?
-Kevin
|