|
PureBytes Links
Trading Reference Links
|
Brian,
you wrote:
> I get robust results optimizing over the largest set of data
> possible. I'd
> say back to inception of the S+P is a robust optimization.
I agree with the principle of having a large in-sample period but optimizing
against it all is not good for robustness. Let us apply the generality to
the specific of OddBall with Walk-forward testing and principles of
robustness.
If you go back to the inception of the S&P and optimize the system variable
until **1996**, and THEN run the system from 1997 through the present, I
might consider that to be a *generalized* robustness test. Running
parameters from the inception of the S&P until the *present* is promising
but is not a guarantee of robustness. You will tend to pick the best figures
and the system may fall apart going forward. For instance, when I run
OddBall parameter optimization of 5,3,1, it only returned $3,000 for
2001-present as compared with my gleeful figures for 7,3,1. The difference
between 5-3-1 and 7-3-1 is only two numeric places. Specifically, what
market inefficiency are you trying to capture with the difference of two ( 5
and 7 ) on the length of the rate of change? I do not know. Do you? When I
optimize the rate of change length (the "7" in 7,3,1), I do not see a bell
curve. Instead, I see the 7 as near a steep decline. This is a robustness
issue, the ability to make profits over a wide range of variables. If market
conditions change, how are you going to protect yourself with a cliff only
two numeric places away? I really want to know.
Are you going to periodically reoptimize to solve this "problem"? If so,
then you may wish to walk-forward precisely with THAT kind of testing. In
other words, start with the S&P from inception and optimize on three years
of data and run it against the next year's out-of-sample data. Then divide
your ACTUAL profits with your PROJECTED profits and you will get your
Walk-Forward Efficiency (WFE) ratio. If you have a WFE ratio of 50% by the
time you get to the present, I will more inclined to say that the system is
robust and willing to put money in it. Going forward into the future, I can
roughly expect that the system will return 50% of my optimized results.
However, optimizing against ALL past data and blindly walking forward
exposes the system to breaking down and loss of money.
A question for you: What do you think about the fact that the profits vary
so widely from 5,3,1 to 7,3,1? Perhaps a prudent step to take would be to
move the "7" away from the "5" a little more. Maybe a "9,3,1" is more in the
center of the bell curve and safer, albeit with less profits.
Sincerely,
Wes Williams
> -----Original Message-----
> From: Brian [mailto:blink64@xxxxxxxxx]
> Sent: Tuesday, February 19, 2002 6:51 PM
> To: List, Omega
> Subject: RE: Oddball and the Emperor's New Clothes
>
>
> "may I submit that you do not know the precise parameters to
> use in the future unless your settings are **ROBUST** and yield
> profits (and
> cut losses quickly) in all kinds of market conditions."
>
> I get robust results optimizing over the largest set of data
> possible. I'd
> say back to inception of the S+P is a robust optimization.
>
> -----Original Message-----
> From: Wes Williams [mailto:softexcl@xxxxxxx]
> Sent: Tuesday, February 19, 2002 4:25 PM
> To: omega-list@xxxxxxxxxx
> Subject: Oddball and the Emperor's New Clothes
>
>
> Dear List,
>
> I am one who has tried different variations of Oddball. I also obtain
> wonderful results AFTER optimization. I also see the ecstatic
> results posted
> by others on the list and only wonder if they are fooling themselves into
> thinking that it is a robust system. After optimizing for **tremendous**
> profits, I wonder if we are all not like those in _The Emperor's New
> Clothes_ who stood back and said "Look at the beautiful profits on this
> system" but ignore the Risk of Ruin and apparent lack of robustness.
>
> Can the Oddballer's who marvel at Oddball's beautiful clothes answer this
> elementary question? What happened to the system from 5/21/2000 -
> 9/19/2000?
> Running 100 @SP contracts against $ADV with parameters 7,3,1
> produced a loss
> of $4,500,000 and a debt of $1,100,000. You are ruined. Now before you
> reply and say "ahh.. that is the problem... you are using the wrong
> parameters!", may I submit that you do not know the precise parameters to
> use in the future unless your settings are **ROBUST** and yield
> profits (and
> cut losses quickly) in all kinds of market conditions.
>
> Using the "standard" settings may have done well in the past and done well
> since 9/19/2000, but how do you account for 5/21 - 9/19 ?
>
> I would really like to know how you plan to survive this period if we are
> at, say, 1/1/2000. Does it really have any clothes that are robust?
>
> Sincerely,
> Wes Williams
>
>
>
|