[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Comments on Backtesting



PureBytes Links

Trading Reference Links

Jim:
> I'd like to suggest that the user community establish a
> standard that these systems are backtested to for the purpose 
> of including these results in the AFL library. 

TJ:> I wish other people on this group speak up about their 
> proposals so we can find something useful. Maybe a poll would 
> be useful?

Are you 'sure' you want people to speak up? You might want to 
reconsider after I give my opinions :=) What I'm about to say might 
be unpopular but I'll take a risk and speak anyway. 

I believe mechanical backtesting is "mostly" spurious and I don't 
think it is wise to rank AFL's by backtesting results. 

As proof consider well-known swing trader Gary B. Smith. He buys 
breakouts over trendlines. I've modeled his simple system and it 
generates poor returns.... but Mr. Smith's trading has made him a 
millionaire. I'm no millionaire but I've turned in some good trades 
lately using simple divergences. These too fail under backtesting 
but the profits in my account are no less real.

Here's why backtesting underperforms a good trader: stock selection. 
I think Smith and I use the same approach. I use Explorations to 
generate a short list of candidates to trade. I put the results into 
a watch list, select the first stock, then use the down arrow to flip 
thru the charts. If I see a price and volume pattern I "like" then I 
buy. Patterns I "like" are based on accumulated experience from 
numerous trades over the years. When I use AmiBroker I never 
backtest... I have no need or desire to have little white and red 
arrows cluttering up my charts!

The problem with backtesting is that it buys every candidate, even 
the ones with bad charts. Traders outperform the computer because 
the eye and brain are very good at visual pattern recognition and 
integrating other factors such as how the major indexes "feel", 
economic events, sector rotation, etc. Compounding the problem is 
that pattern recognition and external data integration are updated 
daily by my brain, whereas the mechanical system holds the stock 
until a rigid exit is triggered.

If one is serious about mechanical backtesting then there is no need 
for a charting program. Just use a database or spreadsheet with the 
trading algorithms (see the Mechanical Investing threads on Motley 
Fool). Let the database or spreadsheet to tell you when to buy and 
sell. No need to ever look at a chart again.

To sum up: backtesting tests the *computer* as trader not the *human* 
as trader. 

Ranking AFL's by backtesting scores can mislead people. A negative 
return system could generate positive returns in the hands of a good 
trader through careful chart selection. Conversely, a system that is 
labelled the "best" may do poorly in the hands of some traders. 

Now that I've said that, I don't mind if AmiBroker has backtesting if 
most people want that feature. Good programs come with a full chest 
of toys and its up to each user to decide which toys to play with. 
But I hope that the focus of AmiBroker stays primarily with charting.

Regards,

Jim Varney