|
PureBytes Links
Trading Reference Links
|
Very informative, especially the last paragraph, which is what I feel
most people, especially new traders such as myself, end up doing
unknowingly.
Could you please elaborate further on following? Do you mean a number
here or something else?
> First, before any modeling begins. Using judgment of management and
> comparison of trading profiles of many trading runs (real,
simulated, or
> imagined), pick an objective function by which the "goodness" of a
trading
> system will be measured. This is important, it is a personal or
corporate
> judgment, and it should not be subject to optimization.
Thanks.
Jitu
--- In amibroker@xxxxxxxxxxxxxxx, "Howard Bandy" <howardbandy@xxxx>
wrote:
> Greetings --
>
> In my opinion, anything we do in development of trading systems
involves a
> search for a pattern than precedes a profitable trading
opportunity. Any
> time we examine the results of alternative systems, we are involved
in
> searching; and when we select the most promising of those
alternatives, we
> are optimizing. Only a system based on truly random entries and
exits would
> not be the result some optimization. So the question of "should we
> optimize?" is moot -- we have no choice but to optimize.
Consequently, we
> should be aware of our optimization techniques.
>
> Chuck referred to an optimization technique recommendation I made
to the
> company we both worked for in Denver a few years ago. This is a
short
> description of it.
>
> The company is a Commodity Trading Advisor which traded futures, not
> individual stocks, but the procedures are equally valid for both.
>
> When I joined the company, they were using very long data series
when
> developing their models. They used a technique sometimes called
folding or
> jackknifing, where the data was divided into several periods -- say
ten.
> The modeling process made ten passes. During each pass, one period
was held
> back to be used as out-of-sample data, the other nine were used to
select
> the best parameter values. After all ten passes, the results were
gathered
> together and the parameter values that scored best overall were
chosen.
> There are several problems with this method. One is the difficulty
with the
> "ramp up" period at the start of each segment, another is that it
is not
> valid to use older data for out-of-sample testing than was used for
> in-sample development, and another is that the data series were too
long.
> Chuck and I and others had many interesting discussions about how
long the
> in-sample data should be.
>
> My background is strong in both the theory and the practice of
modeling and
> simulation, and includes a great deal of experience with analysis of
> financial time series. I proposed the following method, which I
continue to
> believe is valid.
>
> First, before any modeling begins. Using judgment of management and
> comparison of trading profiles of many trading runs (real,
simulated, or
> imagined), pick an objective function by which the "goodness" of a
trading
> system will be measured. This is important, it is a personal or
corporate
> judgment, and it should not be subject to optimization.
>
> Divide each data series into a sequence of in-sample and out-of-
sample
> periods. The length of the out-of-sample period is
the "reoptimization"
> period. Say there are about ten years of historical data available
> (1/1/1993 through 1/1/2003. Set the in-sample period to two years
and the
> out-of-sample period to one year. Run the following sequence:
Search /
> optimize using 1993 and 1994; pick the "best" model for 1993-1994;
forward
> test this model for 1995 and save the results; step forward one
> reoptimization period and repeat until all the full in-sample
periods have
> been used. The final optimization will have been 2001 and 2002,
with no
> out-of-sample data to test. Ignore all in-sample results!!
Examine the
> concatenated out-of-sample equity curve. If it is acceptable, you
have some
> confidence that the parameters select by the final optimization
(2001 and
> 2002) will be profitable for 2003. No guarantees -- only some
confidence.
>
> How did I pick two years for in-sample and one year for out-of-
sample? That
> was just an example. The method is to set up an automated search
where the
> length of the in-sample period and the length of the out-of-sample
period --
> the reoptimization period -- are variables, and then search through
that
> space.
>
> Trading systems work because they identify inefficiencies in
markets. Every
> profitable trade reduces the inefficiency until, finally, the
trading system
> cannot overcome the frictional forces of commission and slippage.
This is
> the same phenomenon that physicists talk about as entropy.
>
> My feeling -- and it may be different than Chuck's -- is that the
market is
> not only non-stationary, but that the probability that it will
return to a
> previous state is near zero.
>
> Being non-stationary means that market conditions change with
respect to our
> trading systems. If I am modeling a physical process, such as a
chemical
> reaction, I can count on a predictable modelable output for a given
set of
> inputs. If I am modeling a financial time series, the output
following a
> given set of inputs changes over time. If a market were stationary
with
> respect to an RSI oscillator system, I could always buy a rise of
the RSI
> through the 20 percent line, to use a very simplistic example.
>
> I feel that the introduction of microcomputers, trading system
development
> software, inexpensive individual brokerage accounts, and discussion
groups
> such as this one have permanently changed the realm of trading.
One,
> everyone who is interested can afford to buy a computer, run
AmiBroker, and
> design and test trading systems. Two, if someone develops a
profitable
> system and trades it, the profits it takes reduce the potential
profits
> available to anyone else who trades it. Consequently, the
characteristics
> of the market change in a way that moves the market away from that
model
> until that trading system is no longer profitable enough to overcome
> commission and slippage. Three, a new person beginning to study
trading
> system development typically tests a lot of old systems. If one is
found to
> be profitable and they start trading it, the market moves back to
being
> efficient. Consequently, trading systems that used to work, but no
longer
> work, are very unlikely to ever work again.
>
> So, I feel that the in-sample period should be short so that the
market
> conditions do not change much over that period. That is, I am
looking for a
> data series that is stationary relative to my model. The stationary
> relationship must extend beyond the in-sample period far enough
that the
> model will be profitable when used for trading in the out-of-sample
data.
> The length of the extension determines the reoptimization period.
It could
> be years, months, or even one day. Note that the holding period of
a
> typical trade is very much related to the length of both the in-
sample and
> out-of-sample periods. The typical trade should be much shorter
than the
> in-sample period and somewhat shorter than the out-of-sample period.
>
> The important point in all this is that the only results being
analyzed are
> the concatenated out-of-sample trades.
>
> As with all model development, every time I look at the out-of-
sample
> results in any way, I reduce the probability that future trading
results
> will be profitable. That means that I should not perform thousands
of tests
> of model parameters, in-sample periods, and out-of-sample periods,
on the
> same data series and then pick the best model base on my
examination of
> thousands of out-of-sample results. In effect, I will have just
converted
> all those out-of-sample results into in-sample data for another
step in the
> development. That is legitimate, just be aware of what is
happening.
>
> Thanks for listening,
> Howard
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Rent DVDs from home.
Over 14,500 titles. Free Shipping
& No Late Fees. Try Netflix for FREE!
http://us.click.yahoo.com/mk9osC/hP.FAA/3jkFAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|