|
PureBytes Links
Trading Reference Links
|
brian <polar@xxxxxxxxxx> wrote:
> Although I do agree you will blow out if you're under capitalized
> or you testing is over optimzed and shoddy. Drawdowns will happen
> but through proper testing you can get an estimate of their length
> and duration and be ready when they occur. Of course it's a lot of
> work but it should be.
I think this would be a great topic of discussion. Brian and others,
would you be willing to share some of the techniques you use for
testing, predicting drawdowns, etc?
When I develop and test a system, I'm looking for several things:
* Robust parameters. I want system parameters that cause the system
results to increase gradually, peak, then decrease gradually as you
sweep through the parameter values. No "spiky" results, as that
means the system performance is not clearly and cleanly controlled by
the parameter, and you don't know what will happen in the future.
Systems with fewer parameters are preferable, but I generally get
much better results with more inputs, and they usually hold up well
in testing.
* Robust results. I don't want a system that made $1M by making
10,000 $100 trades. For one thing, 10,000 trades is way too much
work. More importantly, $100 trades turn into losers too easily, if
you misjudge your slippage or other costs, or if the market behavior
shifts a little. I'd rather have a system that made $500k with 100
$5000 trades. That way if my results slip by $1000 per trade, I'm
still well ahead. I also look for low drawdown, high ROA, high
Sharpe, etc.
* Long AND short performance. I'm no good at calling the market --
that's why I use systems in the first place -- so I don't want to
trade long-only systems. You can do great with those systems as long
as you don't keep trading them in an extended down market, but if you
trade them at the wrong times you'll probably get killed. I prefer
systems that do well, long and short, in up down & sideways markets.
* Sufficient data to show statistical validity. I don't want to
trade a system that only had 20 trades in its test period. With so
few trades, some market aberrations might have an undue impact on
your results. I try to test on a period that produces at least 100-
150 trades. (For my systems, which usually run on 30-60min bars,
that only requires about 2-3 years of data. But 2 years of 30min
bars is as much data as 28 YEARS of daily bars.) That way I figure
I've hit most market conditions that the system is likely to
encounter in the future.
* Out-of-sample performance. I usually tune the system on an older
period, then try running the system on recent data with the same
settings. If it holds up well, that tells me it will probably hold
up well in the future in real-time trading. Then I retune it on all
the data I have for real-time use.
* Smooth equity curve. I don't like drawdowns. They don't look nice
on the system test report, but they're pure hell to trade through. I
want a system that has produced the smoothest possible growth in the
account. (Upward spikes are OK, though. :-) That reduces my stress
level when I trade it, and also makes it safer to increase leverage.
I use the Sharpe ratio as a measure of this, but mainly I eyeball an
equity curve to see how it looks.
* "Normal" test conditions. The last 6 months have been absolutely
whacko in the S&P and ND, with wild and insane swings and volatility.
Lots of systems make a fortune during this time, but lose money in
more typical conditions. I don't want this period to "swamp out" my
results for the last 2-3 years (making a lousy system look good just
because it did great during the volatile period), so I don't
generally include it in my tuning. If I do, I use it mainly to see
if a good "normal conditions" system will also function well during a
wild volatility spike. I try to make the systems adapt to market
conditions so they can do well no matter what the market throws at
them.
There's more, but that gives you a good idea of what I'm looking for.
Who'd like to add their 2 cents?
Gary
|