[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Bryan Mulholland (Omega-List): system results problems and optimization issues



PureBytes Links

Trading Reference Links

Bryan Mulholland (Omega-List):

Bryan, I've had that problem with some FOREX and futures currency
data and S&P TickData when the decimal point was off or missing, or
the dollar point value of the contract was set wrong, as was the case
when setting up a new "TickData" test file (not their fault). You may
want to look into that and see if changing the price multiplier or
the point value in the "Server Symbol Universe" area fixes the problem.

What data vendor data are you referring to and what time frame is the
data you are referring to? 

======================

Regarding your other system statements, it has been my experience
that most, but not all, systems for sale do not work as advertised,
if at all. The problem typically seems to be two fold. First, the
vendor uses too many parameters and you end up with too many "degrees
of freedom," and thus, the system will NOT work for long, or at all,
on out of sample data. And two, the vendor shows only the in-sample
results or did not do "walk-forward" testing, or does not show those
results because they were not good enough to entice anyone to buy the
system.

However, I have noticed that system vendors seem to be getting
better, at least with their sophistication in presenting the testing
results as robust.  As a result, I tend to think that a few system
vendors are probably doing a decent job of representing their
product. If they can show you account statements then you have some
level of proof, but then, did those results come from the system
being sold? 

Additionally, in most cases, you'll need to factor in $200 per trade
in the S&P 500 (big contract) and $600 per trade in the Big NASDAQ to
more accurately replicate actual execution costs over time. Unless,
of course, the orders are always limit orders, in which case, you
could never be sure that you would get the price you'd want anyway or
be filled on every or any order. 

For my purposes, I use a 2 to 3 month window of intra-day data to
develop the parameters for, say, a three variable parameter intra-day
system. (There can be more global or fixed parameters, but with more
than three or four parameters, serious degradation on out of sample
data will often bring you a reality check.)

I will then walk-forward test the system on the next month of
intra-day data, which, by definition, is out-of-sample data. The test
data window is then moved forward one month and the process is
repeated over the entire database--typically five years for intra-day
data, and 10 to 20 years for daily data. 

The big CTA system traders--from what I have gathered form my
conversations with them and from what I've read, tend to have only
two or three optimizable parameters in their original long term
breakout systems. Some like Dunn Capital have been around for over 25
years and have an approximate 25% compounded annual return, on what
is currently around $1.0 billion dollars.  So, if you plan to be
around for a while, and you uses a system to trade, l ess parameters
might be something to consider when designing or purchasing a system.

I certainly do NOT agree with everything the large CTAs do, such as
necessarily using the same system, perhaps with even the same
parameters to trade all markets. Just as every corporation has its
own corporate culture, which is set by the founder, the time of
incorporation and the character of the top employees, every
individual trader has a unique personality and a success cycle.
Furthermore, we know that every stock, futures or currency market has
its own unique personality in the form of price action. 

That said, most systems will NOT fit most markets most of the time to
an optimal level, if at all. However, and like individual traders,
some systems (traders) will work very well on a certain markets most
of the time, and other systems will work very well on other markets
much of the time. It is this system mix or system to market
combination that every good system trader is looking for; just as is
the correct allocation to various traders and methods in a
proprietary trading firm or for a fund of funds manager. 

Be creative but treat your trading and system work like a business;
the better you manage your (trading) business with goals, planned
action, creativity, constant learning & adjustments, and never give
up determination, the better your eventual profitability and
longevity will be. 

SYSTEM PARAMETER DEVELOPMENT: 

I've developed layered systems for myself and for a FOREX firm I used
to work for, but each layer's parameters was keep to a minimum and
pretty much had to work on its own. This process may seem complicated
at first, but walk though it slowly in the simple example below: 

Let's say you use a retracement system (you want to buy dips in an
uptrend & sell short the rallies in a downtrend) with the following
components:

1) a trend component that uses a short, medium and Long-term filter within 
   it, each having one variable parameter (3 total);
2) a single retracement condition (overbought-oversold, Fibonacci levels or 
   a statistic, etc.) with two optimizable parameters (5 variables so far);
3) a volatility breakout entry component with two variable parameters, one 
   for ATRbars and one for the ATRamount (7 variables so far), and;
4) an initial protective stop (IPS), also with two variable parameters (9 
   parameters total).

First of all, to optimize all 9 parameters at the same time on almost
any length of data worth considering and depending on the complexity
of the algorithm's indicators and rules, even on a decent 1.4 GHz HP
P4, it could take months if not years within TradeStation. 

I recently looked at optimizing something like 7 variable parameters
on something with very sophisticated mathematics just to see how long
it would take; how's 10,000 hours or 416 days at 24 hours per day
sound? Of course, as is often suggested, you could simply optimize
only two or three parameters at a time and come to a rough estimation
in relatively no time, but to keep the degrees of freedom low, lets
whittle down the systems final optimizable parameters to three. 

Here's how to do it:

TO BE CONTINUED..........


Best regards,

Mike Mansfield,