|
PureBytes Links
Trading Reference Links
|
Fred,
Watch out where the huskies go...
Take care,
Steve
<BLOCKQUOTE
>
----- Original Message -----
<DIV
>From:
Fred
To: <A title=amibroker@xxxxxxxxxxxxxxx
href="">amibroker@xxxxxxxxxxxxxxx
Sent: Monday, October 20, 2003 10:45
AM
Subject: Objective functions (was RE:
[amibroker] Re: Optimization -- again)
Don't eat the yellow snow either ...--- In <A
href="">amibroker@xxxxxxxxxxxxxxx,
"CedarCreekTrading" <kernish@x...>
wrote:> Pal,> > For a system to be valid, it must
work on all numbers tested, not just those with certain names and not
others > with different names.> > Try telling Keith
Fitchen that (the author of Aberration, the most successful mechanical
system ever sold ... check out Futures Truth). First, Keith will
tell you that his wildly successful approach DOES NOT work on
equities. Second, Keith will tell you that he only trades a basket
of six commodities. I believe both these principles are directly
contrary to your opinions about optimizing and the selection of issues to
be traded.> > If a system works on Bonds and not on Beans, this
system is curve fitted over a specific set of data (Bonds) and it loses
all statistical validity.> > Wrong, wrong, wrong. If I
have an approach that has worked on Bonds for ten years and it doesn't
work on beans...BFD. Should I abandon a robust approach to trading
Bonds...because I can't make "beans" on Beans? >
> Beware of drinking other people's bath water and whatever you do,
don't drink the Kool Aid . > > Take
care,> > Steve> > > -----
Original Message ----- > From: palsanand
> To: amibroker@xxxxxxxxxxxxxxx > Sent:
Monday, October 20, 2003 9:43 AM> Subject: Objective
functions (was RE: [amibroker] Re: Optimization -- again)> >
> Hi,> > In my mind, curve
fitting means either using different systems for >
different markets, or using different parameters of the same system
> for different markets, and this is not valid technical
analysis.> > Historical testing via computer means
feeding a set of numbers (open, > ow, close prices),
and receiving back an output set of rules that > hopefully
will make money trading. The numbers themselves do not have
> names, and the computer doesn't recognize the difference
> between 'Beans' or 'Bonds'. For a system to be valid, it
must work on > all numbers tested, not just those with
certain names and not others > with different
names.> > If a system works on Bonds and not on
Beans, this system is curve > fitted over a specific set of
data (Bonds) and it loses all > statistical validity. To
believe it will work in the future as it has > worked
in the past is very dangerous.> > Also, different
markets do not have different personalities. Again, >
they are reduced to just being a set of numbers or a bunch of
> algorithms. If a channel breakout (or any other) method
is > successful, then the same parameter must be used for
all the markets, > for the same reasons as above. You
cannot use a 20-day channel in > Silver and a 40-day
channel in Corn, this also falls under the crime > of
curve fitting.> > I therefore take exception to any
system, that either only trades one > specific market
or group of markets, or trades different markets > using
different parameters or rules of the same system. All this
> proves is what has worked best in the past, and this will
usually not > continue to work in the future, as there
is no correlation under this > scenario.>
> This is not specifically written to condemn vendors. This
is a > clarification of my definitions of 'optimizing' and
'curve fitting', > and a warning as to what types of
trading systems may be valid and > what to stay away
from.> > Regards,> >
Pal> --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill"
<dmerrill@xxxx> > wrote:> >
thanks, I'll check it out if I can find it.> >
> > I'm sure I'm ignorant, but how logic or sound
trading principles > can be used>
> to set an MA period (for instance) without examination of past
> history> > escapes me. as does the
distinction between using past history or> >
'experience' to do that and optimization. as does the justification
> for> > seeing optimizations from
one point in time as somehow blessed > above
all> > others.> >
> > dave> > I would
have to refer you to an article published by Futures >
Magazine> > concerning optimization and its
research value in November 20?? by> >
Kent Calhoun.> > > >
Possibly the only way to do it correctly, is to first arrive at a
> set> > of parameters
and algorithm based on logic, experience, or sound>
> trading principals that won't be subject to change. Then do a
walk> > forward with no attempt to
improve results via optimization.> >
> > Regards,> >
> > Pal> >
> > > > --- In
amibroker@xxxxxxxxxxxxxxx, "Dave Merrill"
<dmerrill@xxxx>> >
wrote:> > > Pal, couple
questions/comments.> >
>> > > - are you saying that 30
"occurrences" in any system produces >
95%> > accuracy?>
> > 30 trades? regardless of the market or trading system
rules or > time> >
frame?> > > what's the basis for saying
this?> > >>
> > - could you explain "select stable parameters with an
equity > shift> >
less than> > > the parameter shift after
equity spikes have been eliminated"? I>
> don't> > > understand
what you mean.> > >>
> > - just fyi, your last paragraph seems to be trying to
convince > me> >
that> > > optimizing is good, probably in
response to my asking "if you> > prefer
not to> > > optimize parameters, how do
you set them?". I asked that only> >
because you> > > said, "I prefer a system
to work without optimization", which I>
> thought was> > > a nice
goal, but one I don't understand how to achieve. seems
> that> > you
don't> > > actually intend to avoid
optimization either, since you then> >
discuss how you> > > do
it.> > >>
> > dave> >
> There is a correct method to optimize any system that
is> > > statistically valid,
30 occurrences with 95% accuracy.> >
>> > > The key to
optimization is to select stable parameters with an>
> equity> > >
shift less than the parameter shift after equity spikes have
> been> >
> eliminated. This process creates stability for optimal
> parameters> >
> shifts within the four technical market phases. Parameter
> shift is> >
> always geometric, but equity shift decline relative to
> unstable> >
> parameter selection is usually
exponential.> > >>
> > All systems are optimized to some degree. As
soon as a trader> >
chooses> > > to enter a trade
on the open as opposed to the high/low/close >
of> > day,>
> > he has made a decision as to how a system
should be traded. > Does>
> he> > > know
the high/low/close of day entry is better than the
next> > opening>
> > for an entry? If not why not? A potential
28% difference in> > >
profitability exists for channel system entries between opens
> and> > >
closes.> > >>
> > The purpose of trading is to consistently
make money. This is> > done
by> > > having the best
information available. If a trader does not >
know> > the>
> > best entry for his system, what is he trying
to prove? That > the>
> > system isn't optimized? To lose money
because a trader is> > ignorant
of> > > his systen's best
parameters is foolish.> >
>> > >
Regards,> > >>
> > Pal> >
> --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill"
> <dmerrill@xxxx>>
> > wrote:> >
> > one question pal: if you prefer not to optimize
parameters, > how> >
do> > > you
set> > > > them? or do you
have some kind of trading rules that don't >
have> > time>
> > > constants, trigger levels, etc, that
need to be set?> > >
>> > > >
dave> > >
>> > > > I
thought I might throw in my 2 cents.> >
> >> > >
> Vendors love optimization, because it can generate eye
> popping> >
> > hypothetical profits which has no connection
to real-time> >
trading.> > >
>> > > > I
prefer a system to work without optimization. But if I
> have> >
to> > >
do> > > > it,
I would make sure that the optimization is robust in >
the> > > >
following manner:> > >
>> > > > 1.
The sample size of data should be large enough to >
represent> > >
real-> > > >
time market conditions - bull, bear and sideways
markets.> > >
>> > > > 2.
The look-back period should be as large as possible >
for the> > >
same> > > >
reasons.> > >
>> > > > 3.
The testing of optimizable parameters should be on out
> of> > >
sample> > > >
data using walk-forward analysis.> >
> >> > >
> 4. The Central Limit Theorem says that for a sample to
> assume> >
the> > > >
characteristics of the population, the size of sample >
should> > be>
> > > large. The minimum sample
size should be around 30. But >
since> > an>
> > > uptrend or downtrend can
last for say 50 periods, I would> > have
a> > > >
minimum sample size of 100 periods making sure that the
> full> >
> market> > >
> cycle is there (uptrend, downtrend and
congestion).> > >
>> > > > 5.
The optimizable parameters should be as few as > possible
and> > >
tested> > > >
in a wide variety of markets.> >
> >> > >
> Curve-fitting is like rolling a fair dice with
1/6> > probability of>
> > > getting any number from 1
to 6, rolling it 5 times, > getting>
> #6, 4> > >
out> > > > of
5 times (80%) of time.> > >
>> > > > A
lot of traders fall in the trap of curve-fitting
without> > being>
> > > aware of it. So when
designing a system, it is important >
to> > keep>
> > your> >
> > guard up as far as curve-fitting is
concerned.> > >
>> > > >
Regards,> > >
>> > > >
Pal> > > > ---
In amibroker@xxxxxxxxxxxxxxx, "Gary A. Serkhoshian">
> > > <serkhoshian777@xxxx>
wrote:> > > >
> Fred,> > >
> >> > >
> > Could you narrow-down your idea of a reasonable
sample > size> >
for> > > >
backtests. You've been hinting at rather sizeable >
backtesting> > >
> periods, but would like to put some numbers to it.
Also> > wonder if>
> > > you use # of trades as a
guide versus period of time for> >
> backtesting> >
> > period.> >
> > >> >
> > > Thanks,>
> > > >
Gary> > > >
>> > > >
> Fred <fctonetti@xxxx> wrote:> >
> > > There are a lot of questions and
provacative statements > in>
> your> > >
> post,> > >
> > only one of which from my perspective needs
an> > answer/response.>
> > > >>
> > > > Market behavior will
continually change after that ...> >
> > >> >
> > > Change ? from what ? into what ? I
guess this is the > part I>
> > don't> >
> > > follow. To me there is nothing
new in market behavior > now>
> that> > >
> > didn't exist last month, last year, last decade,
last> > century,>
> > but> >
> > > clearly those that take a short sighted
view of history > and>
> the> > >
> > market action that made up that history will clearly
> never> >
see> > >
it.> > > >
> It's a forest and trees thing ...> >
> > >> >
> > > --- In amibroker@xxxxxxxxxxxxxxx, "Dave
Merrill"> >
<dmerrill@xxxx>> > >
> > wrote:> >
> > > > I'm not trying to be
argumentative, honest (:-)... I'm> >
more> > >
than> > > >
a> > > > >
little> > > >
> > sick of saying the same thing over and over, but I j
> u s> >
t> > > d
o> > > > >
n ' t g> > >
> > > e t i t .>
> > > >
>> > > >
> > ------------------------------> >
> > > >>
> > > > > I fail to see the
huge difference in principle between> >
equity> > > >
> feedback and> > >
> > > backtesting.> >
> > > >>
> > > > > let's start by
assuming that backtesting performance > of
a> > >
system> > > >
> and its> > >
> > > parameters over some period of past data tells
you> > something>
> > > about>
> > > >
its> > > >
> > future performance. it's not a perfect predictor,
but> > it's the>
> > > best>
> > > >
evidence> > >
> > > we have. does this seem like a reasonable
starting > point?> >
what> > > >
> alternative> > >
> > > is there?> >
> > > >>
> > > > > if that's true,
why is it better to do it only once? >
what> > > >
> justification is> > >
> > > there for picking one examination period over
another?> > clearly>
> > > >
market> > > >
> > behavior will change continually after that. don't
we> > need a>
> > way> >
> > of> >
> > > working>
> > > > > that looks at
what's been happening and evolves our> >
response?> > >
> > >> >
> > > > sounds like we examine performance
up to some point > and>
> > adjust,> >
> > > trade with>
> > > > > the best-choice
system and parameters for a while, >
then> > >
examine> > > >
and> > > >
> adjust> > >
> > > again later. make sense? what alternative is
there?> > >
> > >> >
> > > > so then, how often do we
re-examine performance > history?>
> to> > > put
it> > > > >
> differently, how long do we ignore any changes in >
market> > >
dynamics> > >
> > that may> >
> > > > or may not have occurred? why
would intermittently> > refusing
to> > > >
look> > > >
> and> > >
> > > respond improve system performance or
reliability?> > >
> > >> >
> > > > if that needs to be done, why not
have the system > itself>
> do> > >
it,> > > >
as> > > > >
part of> > > >
> > its inherent operation? why is it better for us as
an> > outside>
> > > agent>
> > > > to>
> > > > > periodically run
some separate tests, reach into the> >
> internals of> >
> > > the>
> > > > > system, and
change stuff?> > >
> > >> >
> > > > or should we just continue with
the system and > parameters>
> we> > >
> choose> > >
> > at the> >
> > > > beginning? are they somehow more
valid than what we'd> >
choose> > > >
later,> > > >
> using> > >
> > > the same backtesting methods, but on a different
date> > range of>
> > > data?>
> > > >
>> > > >
> > ------------------------------> >
> > > >>
> > > > > I realize that
even if it seems to make sense >
logically,> > this>
> > all> >
> > a> >
> > > complete>
> > > > > crock if no
systems put together like this even >
backtest> > well,>
> > > > never
mind> > > >
> > forward testing.> >
> > > >>
> > > > > but every time I
think about abandoning this line of> >
> research, it> >
> > > seems like>
> > > > > the first thing
I'd want to do with a new system > would
be> > > (let
me> > > > >
guess),> > > >
> > test and possibly adjust it using data up to some
> date,> >
then> > >
run> > > >
> with it for> > >
> > > a while after that and see if equity growth is
good. > if> > it
is,> > > >
I'd> > > >
> want to> > >
> > > lather, rinse and repeat with other in and out of
> sample> >
> data, to> >
> > > make sure>
> > > > > that wasn't
coincidence.> > >
> > >> >
> > > > sounds way too familiar to be a
completely different> >
animal.> > > >
> >> > >
> > > dave> >
> > > > From: Fred
[mailto:fctonetti@xxxx]> > >
> > >> >
> > > > That IS what I was
trying to say. I suspect because>
> equity> > >
> feed> > >
> > back> >
> > > > is like looking in a
rear view mirror, great for> > letting
us> > >
know> > > >
> > where we were and how we could have adjusted the
> past to> >
> make it> > >
> > > better, but that's about
it.> > > > >
> Yahoo! Groups
Sponsor> > > >
> > > > Send BUG
REPORTS to bugs@xxxx> > Send SUGGESTIONS to
suggest@xxxx> >
-----------------------------------------> >
Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx> > (Web page:
<A
href="">http://groups.yahoo.com/group/amiquote/messages/)>
>
-------------------------------------------->
> Check group FAQ at:> > <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html>
> > > Your use of Yahoo! Groups is
subject to the Yahoo! Terms of > Service.> >
> Yahoo! Groups Sponsor
>
ADVERTISEMENT>
>
> >
> Send BUG REPORTS to bugs@xxxx> Send
SUGGESTIONS to suggest@xxxx>
-----------------------------------------> Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)>
--------------------------------------------> Check group
FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > Your use of Yahoo! Groups is subject to the
Yahoo! Terms of Service.Send
BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Yahoo! Groups Sponsor
ADVERTISEMENT
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
|