|
PureBytes Links
Trading Reference Links
|
Mark,
Instead of a long email thread, I suggest that you will profit better from a
consideration of the book "Design, Testing, and Optimization of Trading
Systems" by Robert Pardo, at:
http://www.amazon.com/exec/obidos/ASIN/0471554464/. Kaufmann also has a
must-read chapter on the subject in his book "Trading Systems That Work:
Building and Evaluating Effective Trading Systems" at:
http://www.amazon.com/exec/obidos/ASIN/007135980X.
The rationale behind the process is that if you have variables, you need to
set them to something. Ideally, we would like to set them to something that
will make us money. Setting them to something that makes money in bear
markets but not bull markets would not do well if market conditions changed
to a bull market in the future, or a sideways market. Walk-Forward testing
gives you an objective manner to measure your own process in how you set
variables.
Basically the process is that you optimize on *past* data (called
"in-sample", or IS data) and then run the strategy on data that was not
included in the optimization (called "out-of-sample", or OS data). If you
make zillions of dollars in your IS optimization but go to zero in OS
testing, the idea is that your strategy (or variables) are not robust, that
is, they do not make money over all kinds of market conditions and you are
at risk.
There are various methodologies in IS-OS testing. One is that you optimize
your strategy against, say 1980-1995 IS data and then run it from 1996 -
present to see how well it does. Does it do better? The same? Worse? Another
approach is to periodically reoptimize every n periods with an IS/OS ratio
of 3:1/ 8:1/ 12:1 (depending on the sensitivity of your system) or, since I
think that Kaufmann's approach is better than Pardo's, be sure you have two
bear market, two bull markets and two sideways markets in your IS period and
run *with those variables* into the OS period. Others run their IS optimized
strategies on (1) bull-market OS data only, (2) bear-market OS data only,
and (3) sideways OS data only and then measure the performance of the
variables in each type of market condition.
Pardo goes into metrics of measuring your Walk-Forward Efficiency.
I write walk-forward optimization routines of my strategies in ESL whenever
variables that may change are involved to test if the strategy puts me at
risk. Advice for your friend: a "generic" program would be exceptionally
time-consuming to write *unless* his strategy/ies are limited in number.
Sincerely,
Wes Williams
> -----Original Message-----
> From: Mark Brown [mailto:markbrown@xxxxxxxxxxxxx]
> Sent: Thursday, February 21, 2002 11:48 AM
> To: Omega List
> Subject: Forward Optimization
>
>
> Hello Omega,
>
> I am wondering if someone who understands forward optimization
> processes could please explain the whole idea of the concept to me.
> What is the goal of it? How do you use it? What features do you need
> that are seemingly unobtainable using currently available software.
> What software do you use, are there and references online? I am doing
> some research for a friend who is building a new software product, and
> he has the bright idea he needs to include this feature.
>
> --
>
> Have Great Day, Mark Brown
>
> °¨¨°şİ[ WWW.MARKBROWN.COM ]İş°¨¨°
>
>
|