|
PureBytes Links
Trading Reference Links
|
Pal,
For a system to be valid, it must work on all
numbers tested, not just those with certain names and not others with
different names.
Try telling Keith Fitchen that (the author of
Aberration, the most successful mechanical system ever sold ... check out
Futures Truth). First, Keith will tell you that his wildly successful
approach DOES NOT work on equities. Second, Keith will tell you that
he only trades a basket of six commodities. I believe both these
principles are directly contrary to your opinions about optimizing and the
selection of issues to be traded.
If a system works on Bonds and not on Beans, this
system is curve fitted over a specific set of data (Bonds) and it loses all
statistical validity.
Wrong, wrong, wrong.
If I have an approach that has worked on Bonds for ten years and it doesn't work
on beans...BFD. Should I abandon a robust approach to trading
Bonds...because I can't make "beans" on
Beans?
Beware of drinking other people's bath water and
whatever you do, don't drink the Kool Aid .
Take care,
Steve
<BLOCKQUOTE
>
----- Original Message -----
<DIV
>From:
palsanand
To: <A title=amibroker@xxxxxxxxxxxxxxx
href="">amibroker@xxxxxxxxxxxxxxx
Sent: Monday, October 20, 2003 9:43
AM
Subject: Objective functions (was RE:
[amibroker] Re: Optimization -- again)
Hi,In my mind, curve fitting means either using
different systems for different markets, or using different parameters of
the same system for different markets, and this is not valid technical
analysis.Historical testing via computer means feeding a set of
numbers (open, ow, close prices), and receiving back an output set of
rules that hopefully will make money trading. The numbers themselves do
not have names, and the computer doesn't recognize the difference
between 'Beans' or 'Bonds'. For a system to be valid, it must work on
all numbers tested, not just those with certain names and not others
with different names.If a system works on Bonds and not on Beans,
this system is curve fitted over a specific set of data (Bonds) and it
loses all statistical validity. To believe it will work in the future as
it has worked in the past is very dangerous.Also, different
markets do not have different personalities. Again, they are reduced to
just being a set of numbers or a bunch of algorithms. If a channel
breakout (or any other) method is successful, then the same parameter must
be used for all the markets, for the same reasons as above. You cannot use
a 20-day channel in Silver and a 40-day channel in Corn, this also falls
under the crime of curve fitting.I therefore take exception to any
system, that either only trades one specific market or group of markets,
or trades different markets using different parameters or rules of the
same system. All this proves is what has worked best in the past, and this
will usually not continue to work in the future, as there is no
correlation under this scenario.This is not specifically written
to condemn vendors. This is a clarification of my definitions of
'optimizing' and 'curve fitting', and a warning as to what types of
trading systems may be valid and what to stay away
from.Regards,Pal--- In amibroker@xxxxxxxxxxxxxxx, "Dave
Merrill" <dmerrill@xxxx> wrote:> thanks, I'll check it out if
I can find it.> > I'm sure I'm ignorant, but how logic or sound
trading principles can be used> to set an MA period (for instance)
without examination of past history> escapes me. as does the
distinction between using past history or> 'experience' to do that and
optimization. as does the justification for> seeing optimizations
from one point in time as somehow blessed above all>
others.> > dave> I would have to refer you to
an article published by Futures Magazine> concerning
optimization and its research value in November 20?? by>
Kent Calhoun.> > Possibly the only way to do it
correctly, is to first arrive at a set> of parameters
and algorithm based on logic, experience, or sound> trading
principals that won't be subject to change. Then do a walk>
forward with no attempt to improve results via optimization.>
> Regards,> > Pal>
> > --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill"
<dmerrill@xxxx>> wrote:> > Pal,
couple questions/comments.> >> > -
are you saying that 30 "occurrences" in any system produces
95%> accuracy?> > 30 trades?
regardless of the market or trading system rules or
time> frame?> > what's the basis
for saying this?> >> > - could you
explain "select stable parameters with an equity shift>
less than> > the parameter shift after equity spikes
have been eliminated"? I> don't> >
understand what you mean.> >> > -
just fyi, your last paragraph seems to be trying to convince
me> that> > optimizing is good,
probably in response to my asking "if you> prefer not
to> > optimize parameters, how do you set them?". I
asked that only> because you> > said,
"I prefer a system to work without optimization", which I>
thought was> > a nice goal, but one I don't understand
how to achieve. seems that> you
don't> > actually intend to avoid optimization either,
since you then> discuss how you> > do
it.> >> > dave>
> There is a correct method to optimize any system that
is> > statistically valid, 30 occurrences
with 95% accuracy.> >>
> The key to optimization is to select stable parameters with
an> equity> > shift less
than the parameter shift after equity spikes have been>
> eliminated. This process creates stability for optimal
parameters> > shifts within the four
technical market phases. Parameter shift is>
> always geometric, but equity shift decline relative to
unstable> > parameter selection is
usually exponential.> >>
> All systems are optimized to some degree. As soon as a
trader> chooses> > to
enter a trade on the open as opposed to the high/low/close
of> day,> > he has
made a decision as to how a system should be traded.
Does> he> > know the
high/low/close of day entry is better than the next>
opening> > for an entry? If not why not? A
potential 28% difference in> > profitability
exists for channel system entries between opens and>
> closes.> >>
> The purpose of trading is to consistently make money. This
is> done by> > having the
best information available. If a trader does not know>
the> > best entry for his system, what is he
trying to prove? That the> > system
isn't optimized? To lose money because a trader is>
ignorant of> > his systen's best parameters
is foolish.> >> >
Regards,> >> >
Pal> > --- In amibroker@xxxxxxxxxxxxxxx,
"Dave Merrill" <dmerrill@xxxx>> >
wrote:> > > one question pal: if you
prefer not to optimize parameters, how>
do> > you set>
> > them? or do you have some kind of trading rules that
don't have> time> >
> constants, trigger levels, etc, that need to be set?>
> >> > >
dave> > >>
> > I thought I might throw in my 2
cents.> > >>
> > Vendors love optimization, because it can
generate eye popping> > >
hypothetical profits which has no connection to real-time>
trading.> > >>
> > I prefer a system to work without
optimization. But if I have> to>
> do> > > it,
I would make sure that the optimization is robust in
the> > > following
manner:> > >>
> > 1. The sample size of data should be large
enough to represent> >
real-> > > time market
conditions - bull, bear and sideways markets.>
> >> > > 2.
The look-back period should be as large as possible for
the> > same>
> > reasons.>
> >> > > 3.
The testing of optimizable parameters should be on out
of> > sample>
> > data using walk-forward
analysis.> > >>
> > 4. The Central Limit Theorem says that for a
sample to assume> the>
> > characteristics of the population, the size
of sample should> be>
> > large. The minimum sample size should be
around 30. But since> an>
> > uptrend or downtrend can last for say 50
periods, I would> have a>
> > minimum sample size of 100 periods making
sure that the full> >
market> > > cycle is there
(uptrend, downtrend and congestion).> >
>> > > 5. The optimizable
parameters should be as few as possible and>
> tested> > >
in a wide variety of markets.> >
>> > > Curve-fitting is
like rolling a fair dice with 1/6> probability
of> > > getting any number
from 1 to 6, rolling it 5 times, getting> #6,
4> > out>
> > of 5 times (80%) of
time.> > >>
> > A lot of traders fall in the trap of
curve-fitting without> being>
> > aware of it. So when designing a system, it
is important to> keep>
> your> > >
guard up as far as curve-fitting is concerned.>
> >> > >
Regards,> > >>
> > Pal> >
> --- In amibroker@xxxxxxxxxxxxxxx, "Gary A.
Serkhoshian"> > >
<serkhoshian777@xxxx> wrote:> >
> > Fred,> >
> >> > >
> Could you narrow-down your idea of a reasonable sample
size> for> >
> backtests. You've been hinting at rather sizeable
backtesting> > > periods,
but would like to put some numbers to it. Also>
wonder if> > > you use # of
trades as a guide versus period of time for>
> backtesting> >
> period.> > >
>> > > >
Thanks,> > > >
Gary> > >
>> > > > Fred
<fctonetti@xxxx> wrote:> >
> > There are a lot of questions and provacative statements
in> your> >
> post,> > >
> only one of which from my perspective needs an>
answer/response.> > >
>> > > > Market
behavior will continually change after that ...>
> > >> >
> > Change ? from what ? into what ? I guess this is the
part I> > don't>
> > > follow. To me there is nothing
new in market behavior now> that>
> > > didn't exist last month, last year,
last decade, last> century,>
> but> > >
> clearly those that take a short sighted view of history
and> the> >
> > market action that made up that history will clearly
never> see> >
it.> > > > It's a forest
and trees thing ...> > >
>> > > > --- In
amibroker@xxxxxxxxxxxxxxx, "Dave Merrill">
<dmerrill@xxxx>> > >
> wrote:> > > > >
I'm not trying to be argumentative, honest (:-)... I'm>
more> > than>
> > a> >
> > little> >
> > > sick of saying the same thing over and over, but
I j u s> t> > d
o> > > > n ' t
g> > > > > e
t i t .> > > >
>> > > > >
------------------------------> >
> > >> >
> > > I fail to see the huge difference in principle
between> equity> >
> > feedback and> >
> > > backtesting.> >
> > >> >
> > > let's start by assuming that backtesting
performance of a> >
system> > > > and
its> > > > > parameters
over some period of past data tells you>
something> > >
about> > > >
its> > > > > future
performance. it's not a perfect predictor, but> it's
the> > >
best> > > >
evidence> > > > > we
have. does this seem like a reasonable starting point?>
what> > > >
alternative> > > > > is
there?> > > >
>> > > > > if that's
true, why is it better to do it only once? what>
> > > justification is>
> > > > there for picking one examination
period over another?> clearly>
> > > market>
> > > > behavior will change continually
after that. don't we> need a>
> way> > >
of> > > >
working> > > > > that
looks at what's been happening and evolves our>
response?> > > >
>> > > > > sounds
like we examine performance up to some point and>
> adjust,> > >
> trade with> > > >
> the best-choice system and parameters for a while,
then> > examine>
> > and> >
> > adjust> >
> > > again later. make sense? what alternative is
there?> > > >
>> > > > > so then,
how often do we re-examine performance history?>
to> > put it>
> > > > differently, how long do we ignore
any changes in market> >
dynamics> > > > that
may> > > > > or may not
have occurred? why would intermittently> refusing
to> > >
look> > > >
and> > > > > respond
improve system performance or reliability?>
> > > >>
> > > > if that needs to be done, why not
have the system itself> do>
> it,> > >
as> > > > part
of> > > > > its
inherent operation? why is it better for us as an>
outside> > >
agent> > > >
to> > > > >
periodically run some separate tests, reach into the>
> internals of> >
> > the> >
> > > system, and change stuff?>
> > > >>
> > > > or should we just continue with
the system and parameters> we>
> > choose> >
> > at the> >
> > > beginning? are they somehow more valid than what
we'd> choose> >
> later,> > >
> using> > > > > the
same backtesting methods, but on a different date> range
of> > >
data?> > > >
>> > > > >
------------------------------> >
> > >> >
> > > I realize that even if it seems to make sense
logically,> this> >
all> > >
a> > > >
complete> > > > > crock
if no systems put together like this even backtest>
well,> > > > never
mind> > > > > forward
testing.> > > >
>> > > > > but every
time I think about abandoning this line of>
> research, it> >
> > seems like> >
> > > the first thing I'd want to do with a new system
would be> > (let me>
> > > guess),>
> > > > test and possibly adjust it using
data up to some date,> then>
> run> > >
> with it for> > > >
> a while after that and see if equity growth is good.
if> it is,> >
> I'd> > >
> want to> > > > >
lather, rinse and repeat with other in and out of
sample> > data, to>
> > > make sure>
> > > > that wasn't
coincidence.> > > >
>> > > > > sounds
way too familiar to be a completely different>
animal.> > > >
>> > > > >
dave> > > >
> From: Fred [mailto:fctonetti@xxxx]>
> > > >>
> > > > That IS what I was
trying to say. I suspect because>
equity> > >
feed> > > >
back> > > >
> is like looking in a rear view mirror, great
for> letting us> >
know> > > >
> where we were and how we could have adjusted the past
to> > make it>
> > > > better, but that's
about it.> >
> Yahoo! Groups
Sponsor> > > > Send BUG REPORTS to
bugs@xxxx> Send SUGGESTIONS to
suggest@xxxx>
-----------------------------------------> Post
AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx> (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)>
--------------------------------------------> Check group
FAQ at:> <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html>
> Your use of Yahoo! Groups is subject to the Yahoo! Terms
of Service.Send
BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Yahoo! Groups Sponsor
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
|