|
PureBytes Links
Trading Reference Links
|
So
according to your last paragraph it is impossbile to develop a system that is
consistently successful since just looking for it voids it. Right?
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>d
<FONT
face=Tahoma size=2>-----Original Message-----From: Howard Bandy
[mailto:howardbandy@xxxxxxxxx] Sent: Thursday, October 16, 2003
2:20 PMTo: amibroker@xxxxxxxxxxxxxxxSubject: [amibroker]
Optimization -- againGreetings --In my
opinion, anything we do in development of trading systems involves asearch
for a pattern than precedes a profitable trading opportunity.
Anytime we examine the results of alternative systems, we are involved
insearching; and when we select the most promising of those alternatives,
weare optimizing. Only a system based on truly random entries and
exits wouldnot be the result some optimization. So the question of
"should weoptimize?" is moot -- we have no choice but to optimize.
Consequently, weshould be aware of our optimization
techniques.Chuck referred to an optimization technique recommendation
I made to thecompany we both worked for in Denver a few years ago.
This is a shortdescription of it.The company is a Commodity
Trading Advisor which traded futures, notindividual stocks, but the
procedures are equally valid for both.When I joined the company, they
were using very long data series whendeveloping their models. They
used a technique sometimes called folding orjackknifing, where the data
was divided into several periods -- say ten.The modeling process made ten
passes. During each pass, one period was heldback to be used as
out-of-sample data, the other nine were used to selectthe best parameter
values. After all ten passes, the results were gatheredtogether and
the parameter values that scored best overall were chosen.There are
several problems with this method. One is the difficulty with
the"ramp up" period at the start of each segment, another is that it is
notvalid to use older data for out-of-sample testing than was used
forin-sample development, and another is that the data series were too
long.Chuck and I and others had many interesting discussions about how
long thein-sample data should be. My background is strong in
both the theory and the practice of modeling andsimulation, and includes a
great deal of experience with analysis offinancial time series. I
proposed the following method, which I continue tobelieve is
valid.First, before any modeling begins. Using judgment of
management andcomparison of trading profiles of many trading runs (real,
simulated, orimagined), pick an objective function by which the "goodness"
of a tradingsystem will be measured. This is important, it is a
personal or corporatejudgment, and it should not be subject to
optimization. Divide each data series into a sequence of
in-sample and out-of-sampleperiods. The length of the out-of-sample
period is the "reoptimization"period. Say there are about ten years
of historical data available(1/1/1993 through 1/1/2003. Set the
in-sample period to two years and theout-of-sample period to one
year. Run the following sequence: Search /optimize using 1993
and 1994; pick the "best" model for 1993-1994; forwardtest this model for
1995 and save the results; step forward onereoptimization period and
repeat until all the full in-sample periods havebeen used. The final
optimization will have been 2001 and 2002, with noout-of-sample data to
test. Ignore all in-sample results!! Examine theconcatenated
out-of-sample equity curve. If it is acceptable, you have
someconfidence that the parameters select by the final optimization (2001
and2002) will be profitable for 2003. No guarantees -- only some
confidence.How did I pick two years for in-sample and one year for
out-of-sample? Thatwas just an example. The method is to set
up an automated search where thelength of the in-sample period and the
length of the out-of-sample period --the reoptimization period -- are
variables, and then search through thatspace. Trading
systems work because they identify inefficiencies in markets.
Everyprofitable trade reduces the inefficiency until, finally, the trading
systemcannot overcome the frictional forces of commission and
slippage. This isthe same phenomenon that physicists talk about as
entropy.My feeling -- and it may be different than Chuck's -- is that
the market isnot only non-stationary, but that the probability that it
will return to aprevious state is near zero. Being
non-stationary means that market conditions change with respect to
ourtrading systems. If I am modeling a physical process, such as a
chemicalreaction, I can count on a predictable modelable output for a
given set ofinputs. If I am modeling a financial time series, the
output following agiven set of inputs changes over time. If a market
were stationary withrespect to an RSI oscillator system, I could always
buy a rise of the RSIthrough the 20 percent line, to use a very simplistic
example. I feel that the introduction of microcomputers, trading
system developmentsoftware, inexpensive individual brokerage accounts, and
discussion groupssuch as this one have permanently changed the realm of
trading. One,everyone who is interested can afford to buy a
computer, run AmiBroker, anddesign and test trading systems. Two, if
someone develops a profitablesystem and trades it, the profits it takes
reduce the potential profitsavailable to anyone else who trades it.
Consequently, the characteristicsof the market change in a way that moves
the market away from that modeluntil that trading system is no longer
profitable enough to overcomecommission and slippage. Three, a new
person beginning to study tradingsystem development typically tests a lot
of old systems. If one is found tobe profitable and they start
trading it, the market moves back to beingefficient. Consequently,
trading systems that used to work, but no longerwork, are very unlikely to
ever work again.So, I feel that the in-sample period should be short
so that the marketconditions do not change much over that period.
That is, I am looking for adata series that is stationary relative to my
model. The stationaryrelationship must extend beyond the in-sample
period far enough that themodel will be profitable when used for trading
in the out-of-sample data.The length of the extension determines the
reoptimization period. It couldbe years, months, or even one
day. Note that the holding period of atypical trade is very much
related to the length of both the in-sample andout-of-sample
periods. The typical trade should be much shorter than thein-sample
period and somewhat shorter than the out-of-sample period.The
important point in all this is that the only results being analyzed arethe
concatenated out-of-sample trades.As with all model development, every
time I look at the out-of-sampleresults in any way, I reduce the
probability that future trading resultswill be profitable. That
means that I should not perform thousands of testsof model parameters,
in-sample periods, and out-of-sample periods, on thesame data series and
then pick the best model base on my examination ofthousands of
out-of-sample results. In effect, I will have just convertedall
those out-of-sample results into in-sample data for another step in
thedevelopment. That is legitimate, just be aware of what is
happening.Thanks for listening,HowardSend
BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Yahoo! Groups Sponsor
ADVERTISEMENT
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
|