|
PureBytes Links
Trading Reference Links
|
Hi,
In my mind, curve fitting means either using different systems for
different markets, or using different parameters of the same system
for different markets, and this is not valid technical analysis.
Historical testing via computer means feeding a set of numbers (open,
ow, close prices), and receiving back an output set of rules that
hopefully will make money trading. The numbers themselves do not have
names, and the computer doesn't recognize the difference
between 'Beans' or 'Bonds'. For a system to be valid, it must work on
all numbers tested, not just those with certain names and not others
with different names.
If a system works on Bonds and not on Beans, this system is curve
fitted over a specific set of data (Bonds) and it loses all
statistical validity. To believe it will work in the future as it has
worked in the past is very dangerous.
Also, different markets do not have different personalities. Again,
they are reduced to just being a set of numbers or a bunch of
algorithms. If a channel breakout (or any other) method is
successful, then the same parameter must be used for all the markets,
for the same reasons as above. You cannot use a 20-day channel in
Silver and a 40-day channel in Corn, this also falls under the crime
of curve fitting.
I therefore take exception to any system, that either only trades one
specific market or group of markets, or trades different markets
using different parameters or rules of the same system. All this
proves is what has worked best in the past, and this will usually not
continue to work in the future, as there is no correlation under this
scenario.
This is not specifically written to condemn vendors. This is a
clarification of my definitions of 'optimizing' and 'curve fitting',
and a warning as to what types of trading systems may be valid and
what to stay away from.
Regards,
Pal
--- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx>
wrote:
> thanks, I'll check it out if I can find it.
>
> I'm sure I'm ignorant, but how logic or sound trading principles
can be used
> to set an MA period (for instance) without examination of past
history
> escapes me. as does the distinction between using past history or
> 'experience' to do that and optimization. as does the justification
for
> seeing optimizations from one point in time as somehow blessed
above all
> others.
>
> dave
> I would have to refer you to an article published by Futures
Magazine
> concerning optimization and its research value in November 20?? by
> Kent Calhoun.
>
> Possibly the only way to do it correctly, is to first arrive at a
set
> of parameters and algorithm based on logic, experience, or sound
> trading principals that won't be subject to change. Then do a walk
> forward with no attempt to improve results via optimization.
>
> Regards,
>
> Pal
>
>
> --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx>
> wrote:
> > Pal, couple questions/comments.
> >
> > - are you saying that 30 "occurrences" in any system produces
95%
> accuracy?
> > 30 trades? regardless of the market or trading system rules or
time
> frame?
> > what's the basis for saying this?
> >
> > - could you explain "select stable parameters with an equity
shift
> less than
> > the parameter shift after equity spikes have been eliminated"? I
> don't
> > understand what you mean.
> >
> > - just fyi, your last paragraph seems to be trying to convince
me
> that
> > optimizing is good, probably in response to my asking "if you
> prefer not to
> > optimize parameters, how do you set them?". I asked that only
> because you
> > said, "I prefer a system to work without optimization", which I
> thought was
> > a nice goal, but one I don't understand how to achieve. seems
that
> you don't
> > actually intend to avoid optimization either, since you then
> discuss how you
> > do it.
> >
> > dave
> > There is a correct method to optimize any system that is
> > statistically valid, 30 occurrences with 95% accuracy.
> >
> > The key to optimization is to select stable parameters with an
> equity
> > shift less than the parameter shift after equity spikes have
been
> > eliminated. This process creates stability for optimal
parameters
> > shifts within the four technical market phases. Parameter
shift is
> > always geometric, but equity shift decline relative to
unstable
> > parameter selection is usually exponential.
> >
> > All systems are optimized to some degree. As soon as a trader
> chooses
> > to enter a trade on the open as opposed to the high/low/close
of
> day,
> > he has made a decision as to how a system should be traded.
Does
> he
> > know the high/low/close of day entry is better than the next
> opening
> > for an entry? If not why not? A potential 28% difference in
> > profitability exists for channel system entries between opens
and
> > closes.
> >
> > The purpose of trading is to consistently make money. This is
> done by
> > having the best information available. If a trader does not
know
> the
> > best entry for his system, what is he trying to prove? That
the
> > system isn't optimized? To lose money because a trader is
> ignorant of
> > his systen's best parameters is foolish.
> >
> > Regards,
> >
> > Pal
> > --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill"
<dmerrill@xxxx>
> > wrote:
> > > one question pal: if you prefer not to optimize parameters,
how
> do
> > you set
> > > them? or do you have some kind of trading rules that don't
have
> time
> > > constants, trigger levels, etc, that need to be set?
> > >
> > > dave
> > >
> > > I thought I might throw in my 2 cents.
> > >
> > > Vendors love optimization, because it can generate eye
popping
> > > hypothetical profits which has no connection to real-time
> trading.
> > >
> > > I prefer a system to work without optimization. But if I
have
> to
> > do
> > > it, I would make sure that the optimization is robust in
the
> > > following manner:
> > >
> > > 1. The sample size of data should be large enough to
represent
> > real-
> > > time market conditions - bull, bear and sideways markets.
> > >
> > > 2. The look-back period should be as large as possible
for the
> > same
> > > reasons.
> > >
> > > 3. The testing of optimizable parameters should be on out
of
> > sample
> > > data using walk-forward analysis.
> > >
> > > 4. The Central Limit Theorem says that for a sample to
assume
> the
> > > characteristics of the population, the size of sample
should
> be
> > > large. The minimum sample size should be around 30. But
since
> an
> > > uptrend or downtrend can last for say 50 periods, I would
> have a
> > > minimum sample size of 100 periods making sure that the
full
> > market
> > > cycle is there (uptrend, downtrend and congestion).
> > >
> > > 5. The optimizable parameters should be as few as
possible and
> > tested
> > > in a wide variety of markets.
> > >
> > > Curve-fitting is like rolling a fair dice with 1/6
> probability of
> > > getting any number from 1 to 6, rolling it 5 times,
getting
> #6, 4
> > out
> > > of 5 times (80%) of time.
> > >
> > > A lot of traders fall in the trap of curve-fitting without
> being
> > > aware of it. So when designing a system, it is important
to
> keep
> > your
> > > guard up as far as curve-fitting is concerned.
> > >
> > > Regards,
> > >
> > > Pal
> > > --- In amibroker@xxxxxxxxxxxxxxx, "Gary A. Serkhoshian"
> > > <serkhoshian777@xxxx> wrote:
> > > > Fred,
> > > >
> > > > Could you narrow-down your idea of a reasonable sample
size
> for
> > > backtests. You've been hinting at rather sizeable
backtesting
> > > periods, but would like to put some numbers to it. Also
> wonder if
> > > you use # of trades as a guide versus period of time for
> > backtesting
> > > period.
> > > >
> > > > Thanks,
> > > > Gary
> > > >
> > > > Fred <fctonetti@xxxx> wrote:
> > > > There are a lot of questions and provacative statements
in
> your
> > > post,
> > > > only one of which from my perspective needs an
> answer/response.
> > > >
> > > > Market behavior will continually change after that ...
> > > >
> > > > Change ? from what ? into what ? I guess this is the
part I
> > don't
> > > > follow. To me there is nothing new in market behavior
now
> that
> > > > didn't exist last month, last year, last decade, last
> century,
> > but
> > > > clearly those that take a short sighted view of history
and
> the
> > > > market action that made up that history will clearly
never
> see
> > it.
> > > > It's a forest and trees thing ...
> > > >
> > > > --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill"
> <dmerrill@xxxx>
> > > > wrote:
> > > > > I'm not trying to be argumentative, honest (:-)... I'm
> more
> > than
> > > a
> > > > little
> > > > > sick of saying the same thing over and over, but I j
u s
> t
> > d o
> > > > n ' t g
> > > > > e t i t .
> > > > >
> > > > > ------------------------------
> > > > >
> > > > > I fail to see the huge difference in principle between
> equity
> > > > feedback and
> > > > > backtesting.
> > > > >
> > > > > let's start by assuming that backtesting performance
of a
> > system
> > > > and its
> > > > > parameters over some period of past data tells you
> something
> > > about
> > > > its
> > > > > future performance. it's not a perfect predictor, but
> it's the
> > > best
> > > > evidence
> > > > > we have. does this seem like a reasonable starting
point?
> what
> > > > alternative
> > > > > is there?
> > > > >
> > > > > if that's true, why is it better to do it only once?
what
> > > > justification is
> > > > > there for picking one examination period over another?
> clearly
> > > > market
> > > > > behavior will change continually after that. don't we
> need a
> > way
> > > of
> > > > working
> > > > > that looks at what's been happening and evolves our
> response?
> > > > >
> > > > > sounds like we examine performance up to some point
and
> > adjust,
> > > > trade with
> > > > > the best-choice system and parameters for a while,
then
> > examine
> > > and
> > > > adjust
> > > > > again later. make sense? what alternative is there?
> > > > >
> > > > > so then, how often do we re-examine performance
history?
> to
> > put it
> > > > > differently, how long do we ignore any changes in
market
> > dynamics
> > > > that may
> > > > > or may not have occurred? why would intermittently
> refusing to
> > > look
> > > > and
> > > > > respond improve system performance or reliability?
> > > > >
> > > > > if that needs to be done, why not have the system
itself
> do
> > it,
> > > as
> > > > part of
> > > > > its inherent operation? why is it better for us as an
> outside
> > > agent
> > > > to
> > > > > periodically run some separate tests, reach into the
> > internals of
> > > > the
> > > > > system, and change stuff?
> > > > >
> > > > > or should we just continue with the system and
parameters
> we
> > > choose
> > > > at the
> > > > > beginning? are they somehow more valid than what we'd
> choose
> > > later,
> > > > using
> > > > > the same backtesting methods, but on a different date
> range of
> > > data?
> > > > >
> > > > > ------------------------------
> > > > >
> > > > > I realize that even if it seems to make sense
logically,
> this
> > all
> > > a
> > > > complete
> > > > > crock if no systems put together like this even
backtest
> well,
> > > > never mind
> > > > > forward testing.
> > > > >
> > > > > but every time I think about abandoning this line of
> > research, it
> > > > seems like
> > > > > the first thing I'd want to do with a new system
would be
> > (let me
> > > > guess),
> > > > > test and possibly adjust it using data up to some
date,
> then
> > run
> > > > with it for
> > > > > a while after that and see if equity growth is good.
if
> it is,
> > > I'd
> > > > want to
> > > > > lather, rinse and repeat with other in and out of
sample
> > data, to
> > > > make sure
> > > > > that wasn't coincidence.
> > > > >
> > > > > sounds way too familiar to be a completely different
> animal.
> > > > >
> > > > > dave
> > > > > From: Fred [mailto:fctonetti@x...]
> > > > >
> > > > > That IS what I was trying to say. I suspect because
> equity
> > > feed
> > > > back
> > > > > is like looking in a rear view mirror, great for
> letting us
> > know
> > > > > where we were and how we could have adjusted the
past to
> > make it
> > > > > better, but that's about it.
>
>
> Yahoo! Groups Sponsor
>
>
>
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
>
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of
Service.
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Rent DVDs from home.
Over 14,500 titles. Free Shipping
& No Late Fees. Try Netflix for FREE!
http://us.click.yahoo.com/mk9osC/hP.FAA/3jkFAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|