|
PureBytes Links
Trading Reference Links
|
I agree with most of what you're looking for. You need robust
parameters. The results shouldn't vary markedly if you change the value
a small percentage. I also look for robustness across product sectors.
I still debate with myself over whether or not different products should
be traded using different systems. I have finally settled on an group by
group approach. I want to see robust results across every product in a
class. For example, in short interest rates I want to see profitability
across euros, bas, shirt sterling, euribor, euroswiss, bank bills, etc.
I then want to see proftiability across crude, kerosene, natgas, etc. I
do use different values and time frames other products but the same
overall approach is taken in every sector. I keep the parameters to a
minimum. If I was long on Oct 87 I don't want to tweak the system so
that I was short. Know you were short and then take that risk into
account when building your portfolio. You need to know full well how you
will be positioned in the future.
I also feel you need to use as much data as possible. The basis of
systematic trading is that history repeats itself so if you only focus on
a small portion of the history you only hurting yourself. Bonds may
never return to their volatile 3pt ranges of the early 80's but you never
know. The point is to be prepared for both volatile and flat periods.
As for drawdowns I look at the portfolio as a whole. I will also take
out the best performing product over a time frame and then recheck the
drawdown stats. You also need to use pessimistic assumptions regarding
slippage - especially if you trade on stops. You should also compute the
recovery time from the drawdowns to get an idea how long they last.
Having a large enough sample period and knowing how bad things get at
their worst and how long it takes to recover is important. Then you also
need to realize that at some point in the future you will exceed this
worst case scenario. I keep sufficient capital on hand so that a
drawdown would have to exceed my worst case drawdown by a factor of 12 to
knock me out. You should be trading bonds w/a 5k account. (I don't day
trade so please don't bombard me with all the day trading posts about how
you can return 5k am month with a 5k account. If you can I have a very
lucrative position waiting for you.)
In the end you have to observe the markets, get a feel for how they trade
and then program and test out your ideas. You'll be suprised how much of
what's out there is garbage but you can also find some solid approaches.
Gary Fritz wrote:
> brian <polar@xxxxxxxxxx> wrote:
> > Although I do agree you will blow out if you're under capitalized
> > or you testing is over optimzed and shoddy. Drawdowns will happen
> > but through proper testing you can get an estimate of their length
> > and duration and be ready when they occur. Of course it's a lot of
> > work but it should be.
>
> I think this would be a great topic of discussion. Brian and others,
> would you be willing to share some of the techniques you use for
> testing, predicting drawdowns, etc?
>
> When I develop and test a system, I'm looking for several things:
>
> * Robust parameters. I want system parameters that cause the system
> results to increase gradually, peak, then decrease gradually as you
> sweep through the parameter values. No "spiky" results, as that
> means the system performance is not clearly and cleanly controlled by
> the parameter, and you don't know what will happen in the future.
> Systems with fewer parameters are preferable, but I generally get
> much better results with more inputs, and they usually hold up well
> in testing.
>
> * Robust results. I don't want a system that made $1M by making
> 10,000 $100 trades. For one thing, 10,000 trades is way too much
> work. More importantly, $100 trades turn into losers too easily, if
> you misjudge your slippage or other costs, or if the market behavior
> shifts a little. I'd rather have a system that made $500k with 100
> $5000 trades. That way if my results slip by $1000 per trade, I'm
> still well ahead. I also look for low drawdown, high ROA, high
> Sharpe, etc.
>
> * Long AND short performance. I'm no good at calling the market --
> that's why I use systems in the first place -- so I don't want to
> trade long-only systems. You can do great with those systems as long
> as you don't keep trading them in an extended down market, but if you
> trade them at the wrong times you'll probably get killed. I prefer
> systems that do well, long and short, in up down & sideways markets.
>
> * Sufficient data to show statistical validity. I don't want to
> trade a system that only had 20 trades in its test period. With so
> few trades, some market aberrations might have an undue impact on
> your results. I try to test on a period that produces at least 100-
> 150 trades. (For my systems, which usually run on 30-60min bars,
> that only requires about 2-3 years of data. But 2 years of 30min
> bars is as much data as 28 YEARS of daily bars.) That way I figure
> I've hit most market conditions that the system is likely to
> encounter in the future.
>
> * Out-of-sample performance. I usually tune the system on an older
> period, then try running the system on recent data with the same
> settings. If it holds up well, that tells me it will probably hold
> up well in the future in real-time trading. Then I retune it on all
> the data I have for real-time use.
>
> * Smooth equity curve. I don't like drawdowns. They don't look nice
> on the system test report, but they're pure hell to trade through. I
> want a system that has produced the smoothest possible growth in the
> account. (Upward spikes are OK, though. :-) That reduces my stress
> level when I trade it, and also makes it safer to increase leverage.
> I use the Sharpe ratio as a measure of this, but mainly I eyeball an
> equity curve to see how it looks.
>
> * "Normal" test conditions. The last 6 months have been absolutely
> whacko in the S&P and ND, with wild and insane swings and volatility.
> Lots of systems make a fortune during this time, but lose money in
> more typical conditions. I don't want this period to "swamp out" my
> results for the last 2-3 years (making a lousy system look good just
> because it did great during the volatile period), so I don't
> generally include it in my tuning. If I do, I use it mainly to see
> if a good "normal conditions" system will also function well during a
> wild volatility spike. I try to make the systems adapt to market
> conditions so they can do well no matter what the market throws at
> them.
>
> There's more, but that gives you a good idea of what I'm looking for.
> Who'd like to add their 2 cents?
>
> Gary
|