|
PureBytes Links
Trading Reference Links
|
And by the same token I would think you'd agree that for a variety of
reasons, futures to a large degree is a very different ball game that
lends itself to a variety of treatments that by and large stocks do
not.
PS ... Distributed processing is hardly anything new ...
--- In amibroker@xxxxxxxxxxxxxxx, "Chuck Rademacher"
<chuck_rademacher@x> wrote:
> Fred,
>
> I couldn't help but reply to your question below.
>
> IMO, the software that Howard and I used when we worked together was
> "capable" of doing the automated task of optimizing and selecting
parameters
> to trade going forward. It was (and is) being used by a fairly
successful
> futures fund in Denver. This particular fund has been ranked in
the
> top-ten based on several different criteria over its 15 years of
operation.
> I still work for the company, but Howard has moved on to do other
things.
>
> Before addressing the issue you raised, I have to tell you a bit
about this
> software. It was written by a fella' by the name of Jim Yonan,
with a lot
> of input from Howard and others working for this particular
company. To
> me, the most interesting thing about it was the way it could
distribute
> processing over an entire network of PC's. We have about 20 PC's
in the
> office and some in the homes of staff members. I have 13 PC's in
my home
> here in New Zealand. The software is capable of sending out small
batches
> of optimization runs to any number of PC's and integrating the
results as
> each batch finishes. It's truly amazing to watch!
>
> The software has (if I remember correctly) over 100 objective
functions by
> which out-of-sample results can be measured. Howard actually
created most
> of these functions and they are excellent. Unfortunately, a
humanoid has
> to decide which objective function to use and which results
look "best".
> This is where (IMO) the problem arises. While the fund is still
making
> nice profits for its clients, it could be doing a lot better with
the right
> person driving the research software. I believe that parameters
are chosen
> based on too short of learning period. More importantly, too few
trades are
> used when determing parameters going forward.
>
> So, I believe the answer to your question is that such software
exists. It
> has been sold to other fund managers, but the price tag is over $1
million
> so it isn't going to help the average punter. But the concept
can be
> fairly easily re-created by a couple of good programmers in a month
or two.
>
> BTW, this particular futures fund is 100% mechancal. The trading
software
> monitors every tick in 47 different futures markets and
automatically
> generates orders. Orders are either printed out for a dealer to
call into
> the floor or, in some cases, go straight to the floor
electronically.
>
> The manager of the fund is about to launch a U.S. based stock hedge
fund
> using similar concepts.
> -----Original Message-----
> From: Fred [mailto:fctonetti@x...]
> Sent: Friday, October 17, 2003 11:01 AM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: [amibroker] Re: Optimization -- again
>
>
> Walk forward testing and optimization is a great concept in theory
> and although I've seen lots of ideas for how to set it up and be
used
> over the years, unfortunately I've yet to see anyone demonstrate
that
> it actually works over a variety of market conditions. There are
I'm
> sure lots of reasons for this which I won't delve into here but
the
> question remains, has anyone actually seen this put into practice
> where the result has been a viable system to use in mechanical
> trading ? If so can you please point at something that could be
> looked at objectively that doesn't reside in a black box ?
>
> --- In amibroker@xxxxxxxxxxxxxxx, "Howard Bandy"
<howardbandy@xxxx>
> wrote:
> > Greetings --
> >
> > In my opinion, anything we do in development of trading systems
> involves a
> > search for a pattern than precedes a profitable trading
> opportunity. Any
> > time we examine the results of alternative systems, we are
involved
> in
> > searching; and when we select the most promising of those
> alternatives, we
> > are optimizing. Only a system based on truly random entries and
> exits would
> > not be the result some optimization. So the question
of "should we
> > optimize?" is moot -- we have no choice but to optimize.
> Consequently, we
> > should be aware of our optimization techniques.
> >
> > Chuck referred to an optimization technique recommendation I
made
> to the
> > company we both worked for in Denver a few years ago. This is a
> short
> > description of it.
> >
> > The company is a Commodity Trading Advisor which traded
futures, not
> > individual stocks, but the procedures are equally valid for
both.
> >
> > When I joined the company, they were using very long data series
> when
> > developing their models. They used a technique sometimes called
> folding or
> > jackknifing, where the data was divided into several periods --
say
> ten.
> > The modeling process made ten passes. During each pass, one
period
> was held
> > back to be used as out-of-sample data, the other nine were used
to
> select
> > the best parameter values. After all ten passes, the results
were
> gathered
> > together and the parameter values that scored best overall were
> chosen.
> > There are several problems with this method. One is the
difficulty
> with the
> > "ramp up" period at the start of each segment, another is that
it
> is not
> > valid to use older data for out-of-sample testing than was used
for
> > in-sample development, and another is that the data series were
too
> long.
> > Chuck and I and others had many interesting discussions about
how
> long the
> > in-sample data should be.
> >
> > My background is strong in both the theory and the practice of
> modeling and
> > simulation, and includes a great deal of experience with
analysis of
> > financial time series. I proposed the following method, which I
> continue to
> > believe is valid.
> >
> > First, before any modeling begins. Using judgment of
management and
> > comparison of trading profiles of many trading runs (real,
> simulated, or
> > imagined), pick an objective function by which the "goodness"
of a
> trading
> > system will be measured. This is important, it is a personal or
> corporate
> > judgment, and it should not be subject to optimization.
> >
> > Divide each data series into a sequence of in-sample and out-of-
> sample
> > periods. The length of the out-of-sample period is
> the "reoptimization"
> > period. Say there are about ten years of historical data
available
> > (1/1/1993 through 1/1/2003. Set the in-sample period to two
years
> and the
> > out-of-sample period to one year. Run the following sequence:
> Search /
> > optimize using 1993 and 1994; pick the "best" model for 1993-
1994;
> forward
> > test this model for 1995 and save the results; step forward one
> > reoptimization period and repeat until all the full in-sample
> periods have
> > been used. The final optimization will have been 2001 and 2002,
> with no
> > out-of-sample data to test. Ignore all in-sample results!!
> Examine the
> > concatenated out-of-sample equity curve. If it is acceptable,
you
> have some
> > confidence that the parameters select by the final optimization
> (2001 and
> > 2002) will be profitable for 2003. No guarantees -- only some
> confidence.
> >
> > How did I pick two years for in-sample and one year for out-of-
> sample? That
> > was just an example. The method is to set up an automated
search
> where the
> > length of the in-sample period and the length of the out-of-
sample
> period --
> > the reoptimization period -- are variables, and then search
through
> that
> > space.
> >
> > Trading systems work because they identify inefficiencies in
> markets. Every
> > profitable trade reduces the inefficiency until, finally, the
> trading system
> > cannot overcome the frictional forces of commission and
slippage.
> This is
> > the same phenomenon that physicists talk about as entropy.
> >
> > My feeling -- and it may be different than Chuck's -- is that
the
> market is
> > not only non-stationary, but that the probability that it will
> return to a
> > previous state is near zero.
> >
> > Being non-stationary means that market conditions change with
> respect to our
> > trading systems. If I am modeling a physical process, such as a
> chemical
> > reaction, I can count on a predictable modelable output for a
given
> set of
> > inputs. If I am modeling a financial time series, the output
> following a
> > given set of inputs changes over time. If a market were
stationary
> with
> > respect to an RSI oscillator system, I could always buy a rise
of
> the RSI
> > through the 20 percent line, to use a very simplistic example.
> >
> > I feel that the introduction of microcomputers, trading system
> development
> > software, inexpensive individual brokerage accounts, and
discussion
> groups
> > such as this one have permanently changed the realm of trading.
> One,
> > everyone who is interested can afford to buy a computer, run
> AmiBroker, and
> > design and test trading systems. Two, if someone develops a
> profitable
> > system and trades it, the profits it takes reduce the potential
> profits
> > available to anyone else who trades it. Consequently, the
> characteristics
> > of the market change in a way that moves the market away from
that
> model
> > until that trading system is no longer profitable enough to
overcome
> > commission and slippage. Three, a new person beginning to study
> trading
> > system development typically tests a lot of old systems. If
one is
> found to
> > be profitable and they start trading it, the market moves back
to
> being
> > efficient. Consequently, trading systems that used to work,
but no
> longer
> > work, are very unlikely to ever work again.
> >
> > So, I feel that the in-sample period should be short so that the
> market
> > conditions do not change much over that period. That is, I am
> looking for a
> > data series that is stationary relative to my model. The
stationary
> > relationship must extend beyond the in-sample period far enough
> that the
> > model will be profitable when used for trading in the out-of-
sample
> data.
> > The length of the extension determines the reoptimization
period.
> It could
> > be years, months, or even one day. Note that the holding
period of
> a
> > typical trade is very much related to the length of both the in-
> sample and
> > out-of-sample periods. The typical trade should be much shorter
> than the
> > in-sample period and somewhat shorter than the out-of-sample
period.
> >
> > The important point in all this is that the only results being
> analyzed are
> > the concatenated out-of-sample trades.
> >
> > As with all model development, every time I look at the out-of-
> sample
> > results in any way, I reduce the probability that future trading
> results
> > will be profitable. That means that I should not perform
thousands
> of tests
> > of model parameters, in-sample periods, and out-of-sample
periods,
> on the
> > same data series and then pick the best model base on my
> examination of
> > thousands of out-of-sample results. In effect, I will have just
> converted
> > all those out-of-sample results into in-sample data for another
> step in the
> > development. That is legitimate, just be aware of what is
> happening.
> >
> > Thanks for listening,
> > Howard
>
>
> Yahoo! Groups Sponsor
> ADVERTISEMENT
>
>
>
>
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
>
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of
Service.
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Rent DVDs Online - Over 14,500 titles.
No Late Fees & Free Shipping.
Try Netflix for FREE!
http://us.click.yahoo.com/Tq9otC/XP.FAA/3jkFAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|