Re: Backtesting

[Bob Fulks <bfulks@xxxxxxxxxxxx>]

At 9:21 PM -0400 6/20/02, Ron Hughes wrote:

>                      I have a question I was hoping some of you might be
>willing to answer. I am doing some back-testing in order to optimize the
>inputs on a system that I've developed. The time required to complete this
>process seems rather excessive, five hundred hours. Is there any way to
>speed up the back-testing process, or is the time required simply a function
>of the number of variables in the equation and the limitations of the user's
>hardware.? Your comments would be greatly appreciated.

One additional suggestion is to make the inputs "orthogonal" if at
all possible.

Orthogonal strictly means "at right angles" but refers to being
"independent" such that the variables do not interact. That allows
you to vary each independently and not have a lot of interaction
between them.

For example, if you are optimizing a simple moving average crossover
system, most people might have two inputs: SlowLength and FastLength
for the length of the two moving averages. Then, they might optimize
both over a range of 5 to 50 by 1. That would be 2116 tests.

But this creates many nonsense cases where the FastLength is larger
than the SlowLength. (We know it should always be smaller.)

I would revise this problem as follows.

Define a new variable that is the ratio of the two lengths. Then vary
one length and the ratio separately:

   Input: SlowLength(10), Ratio(0.5);
   Vars: FastLength(SlowLength * Ratio);
   if Average(Close, FastLength) crosses over Average(Close,
      Slowlength) then Buy...;

Now you can optimize on Slowlength and the code will keep the
FastLength a constant sub-multiple of the SlowLength. Once you find a
useful region for SlowLength, you could then optimize on Ratio to
vary FastLength separately.

You might first vary SlowLength from 5 to 50 by 1 and find a useful
region near SlowLength = 20, Then vary Ratio from 0.2 to 0.9 by 0.1
and find a useful region near 0.6. The vary both from 15 to 25 by 1,
and 0.4 to 0.8 by 0.05, respectively, to zero in on the beast values.

That is 46 + 8 + 11*9 = 153 tests instead of the original 2116 tests.

This is a very simple example but it illustrates how you can keep the
variables varying together in some reasonable way to reduce the
number of tests and make the tests you run more meaningful.

Bob Fulks