|
PureBytes Links
Trading Reference Links
|
<FONT face=Arial color=#0000ff
size=2>Thanks, Steve, for beating me to the punch on this one.
Twilight zone material, for sure.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>.From:
CedarCreekTrading [mailto:kernish@xxxxxxxxxxx]Sent: Monday, October
20, 2003 12:27 PMTo: amibroker@xxxxxxxxxxxxxxx
<BLOCKQUOTE
>
Subject: Re: Objective functions (was RE: [amibroker] Re:
Optimization -- again)
Pal,
For a system to be valid, it must work on all
numbers tested, not just those with certain names and not others with
different names.
Try telling Keith Fitchen that (the author of
Aberration, the most successful mechanical system ever sold ... check out
Futures Truth). First, Keith will tell you that his wildly successful
approach DOES NOT work on equities. Second, Keith will tell you
that he only trades a basket of six commodities. I believe both these
principles are directly contrary to your opinions about optimizing and the
selection of issues to be traded.
If a system works on Bonds and not on Beans,
this system is curve fitted over a specific set of data (Bonds) and it loses
all statistical validity.
Wrong, wrong,
wrong. If I have an approach that has worked on Bonds for ten years and
it doesn't work on beans...BFD. Should I abandon a robust approach to
trading Bonds...because I can't make "beans" on
Beans?
<FONT face=Arial
size=2>
Beware of drinking other people's bath water and
whatever you do, don't drink the Kool Aid .
Take care,
Steve
<BLOCKQUOTE
>
----- Original Message -----
<DIV
>From:
palsanand
To: <A title=amibroker@xxxxxxxxxxxxxxx
href="">amibroker@xxxxxxxxxxxxxxx
Sent: Monday, October 20, 2003 9:43
AM
Subject: Objective functions (was RE:
[amibroker] Re: Optimization -- again)
Hi,In my mind, curve fitting means either using
different systems for different markets, or using different parameters
of the same system for different markets, and this is not valid
technical analysis.Historical testing via computer means feeding a
set of numbers (open, ow, close prices), and receiving back an output
set of rules that hopefully will make money trading. The numbers
themselves do not have names, and the computer doesn't recognize the
difference between 'Beans' or 'Bonds'. For a system to be valid, it must
work on all numbers tested, not just those with certain names and not
others with different names.If a system works on Bonds and not
on Beans, this system is curve fitted over a specific set of data
(Bonds) and it loses all statistical validity. To believe it will work
in the future as it has worked in the past is very
dangerous.Also, different markets do not have different
personalities. Again, they are reduced to just being a set of numbers or
a bunch of algorithms. If a channel breakout (or any other) method is
successful, then the same parameter must be used for all the markets,
for the same reasons as above. You cannot use a 20-day channel in
Silver and a 40-day channel in Corn, this also falls under the crime
of curve fitting.I therefore take exception to any system, that
either only trades one specific market or group of markets, or trades
different markets using different parameters or rules of the same
system. All this proves is what has worked best in the past, and this
will usually not continue to work in the future, as there is no
correlation under this scenario.This is not specifically written
to condemn vendors. This is a clarification of my definitions of
'optimizing' and 'curve fitting', and a warning as to what types of
trading systems may be valid and what to stay away
from.Regards,Pal--- In amibroker@xxxxxxxxxxxxxxx, "Dave
Merrill" <dmerrill@xxxx> wrote:> thanks, I'll check it out
if I can find it.> > I'm sure I'm ignorant, but how logic or
sound trading principles can be used> to set an MA period (for
instance) without examination of past history> escapes me. as
does the distinction between using past history or> 'experience' to
do that and optimization. as does the justification for> seeing
optimizations from one point in time as somehow blessed above
all> others.> > dave> I would have
to refer you to an article published by Futures
Magazine> concerning optimization and its research
value in November 20?? by> Kent Calhoun.>
> Possibly the only way to do it correctly, is to first
arrive at a set> of parameters and algorithm based on
logic, experience, or sound> trading principals that
won't be subject to change. Then do a walk> forward with
no attempt to improve results via optimization.> >
Regards,> > Pal> >
> --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill"
<dmerrill@xxxx>> wrote:> >
Pal, couple questions/comments.> >>
> - are you saying that 30 "occurrences" in any system produces
95%> accuracy?> > 30 trades?
regardless of the market or trading system rules or
time> frame?> > what's the
basis for saying this?> >> > -
could you explain "select stable parameters with an equity
shift> less than> > the
parameter shift after equity spikes have been eliminated"?
I> don't> > understand what you
mean.> >> > - just fyi, your
last paragraph seems to be trying to convince me>
that> > optimizing is good, probably in response to my
asking "if you> prefer not to> >
optimize parameters, how do you set them?". I asked that
only> because you> > said, "I
prefer a system to work without optimization", which I>
thought was> > a nice goal, but one I don't understand
how to achieve. seems that> you
don't> > actually intend to avoid optimization either,
since you then> discuss how you> >
do it.> >> >
dave> > There is a correct method to
optimize any system that is> >
statistically valid, 30 occurrences with 95% accuracy.>
>> > The key to optimization is to
select stable parameters with an>
equity> > shift less than the parameter
shift after equity spikes have been> >
eliminated. This process creates stability for optimal
parameters> > shifts within the four
technical market phases. Parameter shift is>
> always geometric, but equity shift decline relative to
unstable> > parameter selection is
usually exponential.> >>
> All systems are optimized to some degree. As soon as a
trader> chooses> > to
enter a trade on the open as opposed to the high/low/close
of> day,> > he has
made a decision as to how a system should be traded.
Does> he> > know
the high/low/close of day entry is better than the next>
opening> > for an entry? If not why not? A
potential 28% difference in> >
profitability exists for channel system entries between opens
and> > closes.>
>> > The purpose of trading is to
consistently make money. This is> done
by> > having the best information
available. If a trader does not know>
the> > best entry for his system, what is
he trying to prove? That the> > system
isn't optimized? To lose money because a trader is>
ignorant of> > his systen's best
parameters is foolish.> >>
> Regards,> >>
> Pal> > --- In
amibroker@xxxxxxxxxxxxxxx, "Dave Merrill"
<dmerrill@xxxx>> >
wrote:> > > one question pal: if you
prefer not to optimize parameters, how>
do> > you set>
> > them? or do you have some kind of trading rules that
don't have> time> >
> constants, trigger levels, etc, that need to be
set?> > >>
> > dave> >
>> > > I thought I might
throw in my 2 cents.> >
>> > > Vendors love
optimization, because it can generate eye popping>
> > hypothetical profits which has no
connection to real-time> trading.>
> >> > >
I prefer a system to work without optimization. But if I
have> to> >
do> > > it, I would make
sure that the optimization is robust in the>
> > following manner:>
> >> > >
1. The sample size of data should be large enough to
represent> > real->
> > time market conditions - bull, bear and
sideways markets.> >
>> > > 2. The look-back
period should be as large as possible for the>
> same> > >
reasons.> > >>
> > 3. The testing of optimizable parameters
should be on out of> >
sample> > > data using
walk-forward analysis.> >
>> > > 4. The Central
Limit Theorem says that for a sample to assume>
the> > > characteristics of
the population, the size of sample should>
be> > > large. The minimum
sample size should be around 30. But since>
an> > > uptrend or
downtrend can last for say 50 periods, I would> have
a> > > minimum sample size
of 100 periods making sure that the full>
> market> >
> cycle is there (uptrend, downtrend and
congestion).> > >>
> > 5. The optimizable parameters should be as
few as possible and> >
tested> > > in a wide
variety of markets.> >
>> > > Curve-fitting is
like rolling a fair dice with 1/6> probability
of> > > getting any number
from 1 to 6, rolling it 5 times, getting> #6,
4> > out>
> > of 5 times (80%) of
time.> > >>
> > A lot of traders fall in the trap of
curve-fitting without> being>
> > aware of it. So when designing a system,
it is important to> keep>
> your> > >
guard up as far as curve-fitting is concerned.>
> >> > >
Regards,> > >>
> > Pal> >
> --- In amibroker@xxxxxxxxxxxxxxx, "Gary A.
Serkhoshian"> > >
<serkhoshian777@xxxx> wrote:> >
> > Fred,> >
> >> > >
> Could you narrow-down your idea of a reasonable sample
size> for> >
> backtests. You've been hinting at rather sizeable
backtesting> > >
periods, but would like to put some numbers to it.
Also> wonder if> >
> you use # of trades as a guide versus period of time
for> > backtesting>
> > period.>
> > >> >
> > Thanks,> >
> > Gary> >
> >> > >
> Fred <fctonetti@xxxx> wrote:> >
> > There are a lot of questions and provacative
statements in> your>
> > post,> >
> > only one of which from my perspective needs
an> answer/response.> >
> >> > >
> Market behavior will continually change after that
...> > >
>> > > > Change ?
from what ? into what ? I guess this is the part I>
> don't> > >
> follow. To me there is nothing new in market behavior
now> that> >
> > didn't exist last month, last year, last decade,
last> century,> >
but> > > > clearly those
that take a short sighted view of history and>
the> > > > market action
that made up that history will clearly never>
see> > it.>
> > > It's a forest and trees thing
...> > >
>> > > > --- In
amibroker@xxxxxxxxxxxxxxx, "Dave Merrill">
<dmerrill@xxxx>> > >
> wrote:> > > > >
I'm not trying to be argumentative, honest (:-)... I'm>
more> > than>
> > a> >
> > little> >
> > > sick of saying the same thing over and over, but
I j u s> t> >
d o> > > > n '
t g> > > >
> e t i t .> >
> > >> >
> > >
------------------------------> >
> > >> >
> > > I fail to see the huge difference in principle
between> equity> >
> > feedback and> >
> > > backtesting.> >
> > >> >
> > > let's start by assuming that backtesting
performance of a> >
system> > > > and
its> > > > >
parameters over some period of past data tells you>
something> > >
about> > > >
its> > > > > future
performance. it's not a perfect predictor, but> it's
the> > >
best> > > >
evidence> > > > > we
have. does this seem like a reasonable starting
point?> what> >
> > alternative> >
> > > is there?> >
> > >> >
> > > if that's true, why is it better to do it only
once? what> > > >
justification is> > > >
> there for picking one examination period over
another?> clearly> >
> > market> >
> > > behavior will change continually after that.
don't we> need a> >
way> > >
of> > > >
working> > > > > that
looks at what's been happening and evolves our>
response?> > > >
>> > > > > sounds
like we examine performance up to some point and>
> adjust,> >
> > trade with> >
> > > the best-choice system and parameters for a
while, then> >
examine> > >
and> > > >
adjust> > > > > again
later. make sense? what alternative is there?>
> > > >>
> > > > so then, how often do we
re-examine performance history?>
to> > put it>
> > > > differently, how long do we
ignore any changes in market> >
dynamics> > > > that
may> > > > > or may
not have occurred? why would intermittently> refusing
to> > >
look> > > >
and> > > > > respond
improve system performance or reliability?>
> > > >>
> > > > if that needs to be done, why
not have the system itself> do>
> it,> > >
as> > > > part
of> > > > > its
inherent operation? why is it better for us as an>
outside> > >
agent> > > >
to> > > > >
periodically run some separate tests, reach into the>
> internals of> >
> > the> >
> > > system, and change stuff?>
> > > >>
> > > > or should we just continue with
the system and parameters> we>
> > choose>
> > > at the>
> > > > beginning? are they somehow more
valid than what we'd> choose>
> > later,>
> > > using>
> > > > the same backtesting methods,
but on a different date> range of>
> > data?> >
> > >> >
> > >
------------------------------> >
> > >> >
> > > I realize that even if it seems to make sense
logically,> this> >
all> > >
a> > > >
complete> > > > >
crock if no systems put together like this even
backtest> well,> >
> > never mind> >
> > > forward testing.>
> > > >>
> > > > but every time I think about
abandoning this line of> > research,
it> > > > seems
like> > > > > the
first thing I'd want to do with a new system would
be> > (let me>
> > > guess),>
> > > > test and possibly adjust it
using data up to some date,> then>
> run> > >
> with it for> > > >
> a while after that and see if equity growth is good.
if> it is,> >
> I'd> > >
> want to> > > > >
lather, rinse and repeat with other in and out of
sample> > data, to>
> > > make sure>
> > > > that wasn't
coincidence.> > > >
>> > > > > sounds
way too familiar to be a completely different>
animal.> > > >
>> > > > >
dave> > > >
> From: Fred [mailto:fctonetti@xxxx]>
> > > >>
> > > > That IS what I was
trying to say. I suspect because>
equity> > >
feed> > > >
back> > > >
> is like looking in a rear view mirror, great
for> letting us> >
know> > > >
> where we were and how we could have adjusted the past
to> > make it>
> > > > better, but that's
about it.> >
> Yahoo! Groups
Sponsor> > > > Send BUG REPORTS to
bugs@xxxx> Send SUGGESTIONS to
suggest@xxxx>
-----------------------------------------> Post
AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx> (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)>
--------------------------------------------> Check group
FAQ at:> <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html>
> Your use of Yahoo! Groups is subject to the Yahoo!
Terms of Service.Send BUG REPORTS to
bugs@xxxxxxxxxxxxxSend SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page:
<A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Send
BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Yahoo! Groups Sponsor
ADVERTISEMENT
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
|