[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Objective functions (was RE: [amibroker] Re: Optimization -- again)



PureBytes Links

Trading Reference Links

I would have to refer you to an article published by Futures Magazine 
concerning optimization and its research value in November 20?? by 
Kent Calhoun.

Possibly the only way to do it correctly, is to first arrive at a set
of parameters and algorithm based on logic, experience, or sound
trading principals that won't be subject to change. Then do a walk
forward with no attempt to improve results via optimization.

Regards,

Pal 


--- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx> 
wrote:
> Pal, couple questions/comments.
> 
> - are you saying that 30 "occurrences" in any system produces 95% 
accuracy?
> 30 trades? regardless of the market or trading system rules or time 
frame?
> what's the basis for saying this?
> 
> - could you explain "select stable parameters with an equity shift 
less than
> the parameter shift after equity spikes have been eliminated"? I 
don't
> understand what you mean.
> 
> - just fyi, your last paragraph seems to be trying to convince me 
that
> optimizing is good, probably in response to my asking "if you 
prefer not to
> optimize parameters, how do you set them?". I asked that only 
because you
> said, "I prefer a system to work without optimization", which I 
thought was
> a nice goal, but one I don't understand how to achieve. seems that 
you don't
> actually intend to avoid optimization either, since you then 
discuss how you
> do it.
> 
> dave
>   There is a correct method to optimize any system that is
>   statistically valid, 30 occurrences with 95% accuracy.
> 
>   The key to optimization is to select stable parameters with an 
equity
>   shift less than the parameter shift after equity spikes have been
>   eliminated. This process creates stability for optimal parameters
>   shifts within the four technical market phases. Parameter shift is
>   always geometric, but equity shift decline relative to unstable
>   parameter selection is usually exponential.
> 
>   All systems are optimized to some degree. As soon as a trader 
chooses
>   to enter a trade on the open as opposed to the high/low/close of 
day,
>   he has made a decision as to how a system should be traded. Does 
he
>   know the high/low/close of day entry is better than the next 
opening
>   for an entry? If not why not? A potential 28% difference in
>   profitability exists for channel system entries between opens and
>   closes.
> 
>   The purpose of trading is to consistently make money. This is 
done by
>   having the best information available. If a trader does not know 
the
>   best entry for his system, what is he trying to prove? That the
>   system isn't optimized? To lose money because a trader is 
ignorant of
>   his systen's best parameters is foolish.
> 
>   Regards,
> 
>   Pal
>   --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx>
>   wrote:
>   > one question pal: if you prefer not to optimize parameters, how 
do
>   you set
>   > them? or do you have some kind of trading rules that don't have 
time
>   > constants, trigger levels, etc, that need to be set?
>   >
>   > dave
>   >
>   >   I thought I might throw in my 2 cents.
>   >
>   >   Vendors love optimization, because it can generate eye popping
>   >   hypothetical profits which has no connection to real-time 
trading.
>   >
>   >   I prefer a system to work without optimization. But if I have 
to
>   do
>   >   it, I would make sure that the optimization is robust in the
>   >   following manner:
>   >
>   >   1. The sample size of data should be large enough to represent
>   real-
>   >   time market conditions - bull, bear and sideways markets.
>   >
>   >   2. The look-back period should be as large as possible for the
>   same
>   >   reasons.
>   >
>   >   3. The testing of optimizable parameters should be on out of
>   sample
>   >   data using walk-forward analysis.
>   >
>   >   4. The Central Limit Theorem says that for a sample to assume 
the
>   >   characteristics of the population, the size of sample should 
be
>   >   large. The minimum sample size should be around 30. But since 
an
>   >   uptrend or downtrend can last for say 50 periods, I would 
have a
>   >   minimum sample size of 100 periods making sure that the full
>   market
>   >   cycle is there (uptrend, downtrend and congestion).
>   >
>   >   5. The optimizable parameters should be as few as possible and
>   tested
>   >   in a wide variety of markets.
>   >
>   >   Curve-fitting is like rolling a fair dice with 1/6 
probability of
>   >   getting any number from 1 to 6, rolling it 5 times, getting 
#6, 4
>   out
>   >   of 5 times (80%) of time.
>   >
>   >   A lot of traders fall in the trap of curve-fitting without 
being
>   >   aware of it. So when designing a system, it is important to 
keep
>   your
>   >   guard up as far as curve-fitting is concerned.
>   >
>   >   Regards,
>   >
>   >   Pal
>   >   --- In amibroker@xxxxxxxxxxxxxxx, "Gary A. Serkhoshian"
>   >   <serkhoshian777@xxxx> wrote:
>   >   > Fred,
>   >   >
>   >   > Could you narrow-down your idea of a reasonable sample size 
for
>   >   backtests.  You've been hinting at rather sizeable backtesting
>   >   periods, but would like to put some numbers to it.  Also 
wonder if
>   >   you use # of trades as a guide versus period of time for
>   backtesting
>   >   period.
>   >   >
>   >   > Thanks,
>   >   > Gary
>   >   >
>   >   > Fred <fctonetti@xxxx> wrote:
>   >   > There are a lot of questions and provacative statements in 
your
>   >   post,
>   >   > only one of which from my perspective needs an 
answer/response.
>   >   >
>   >   > Market behavior will continually change after that ...
>   >   >
>   >   > Change ? from what ? into what ? I guess this is the part I
>   don't
>   >   > follow.  To me there is nothing new in market behavior now 
that
>   >   > didn't exist last month, last year, last decade, last 
century,
>   but
>   >   > clearly those that take a short sighted view of history and 
the
>   >   > market action that made up that history will clearly never 
see
>   it.
>   >   > It's a forest and trees thing ...
>   >   >
>   >   > --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" 
<dmerrill@xxxx>
>   >   > wrote:
>   >   > > I'm not trying to be argumentative, honest (:-)... I'm 
more
>   than
>   >   a
>   >   > little
>   >   > > sick of saying the same thing over and over, but I  j u s 
t
>   d o
>   >   > n ' t   g
>   >   > > e t   i t .
>   >   > >
>   >   > > ------------------------------
>   >   > >
>   >   > > I fail to see the huge difference in principle between 
equity
>   >   > feedback and
>   >   > > backtesting.
>   >   > >
>   >   > > let's start by assuming that backtesting performance of a
>   system
>   >   > and its
>   >   > > parameters over some period of past data tells you 
something
>   >   about
>   >   > its
>   >   > > future performance. it's not a perfect predictor, but 
it's the
>   >   best
>   >   > evidence
>   >   > > we have. does this seem like a reasonable starting point? 
what
>   >   > alternative
>   >   > > is there?
>   >   > >
>   >   > > if that's true, why is it better to do it only once? what
>   >   > justification is
>   >   > > there for picking one examination period over another? 
clearly
>   >   > market
>   >   > > behavior will change continually after that. don't we 
need a
>   way
>   >   of
>   >   > working
>   >   > > that looks at what's been happening and evolves our 
response?
>   >   > >
>   >   > > sounds like we examine performance up to some point and
>   adjust,
>   >   > trade with
>   >   > > the best-choice system and parameters for a while, then
>   examine
>   >   and
>   >   > adjust
>   >   > > again later. make sense? what alternative is there?
>   >   > >
>   >   > > so then, how often do we re-examine performance history? 
to
>   put it
>   >   > > differently, how long do we ignore any changes in market
>   dynamics
>   >   > that may
>   >   > > or may not have occurred? why would intermittently 
refusing to
>   >   look
>   >   > and
>   >   > > respond improve system performance or reliability?
>   >   > >
>   >   > > if that needs to be done, why not have the system itself 
do
>   it,
>   >   as
>   >   > part of
>   >   > > its inherent operation? why is it better for us as an 
outside
>   >   agent
>   >   > to
>   >   > > periodically run some separate tests, reach into the
>   internals of
>   >   > the
>   >   > > system, and change stuff?
>   >   > >
>   >   > > or should we just continue with the system and parameters 
we
>   >   choose
>   >   > at the
>   >   > > beginning? are they somehow more valid than what we'd 
choose
>   >   later,
>   >   > using
>   >   > > the same backtesting methods, but on a different date 
range of
>   >   data?
>   >   > >
>   >   > > ------------------------------
>   >   > >
>   >   > > I realize that even if it seems to make sense logically, 
this
>   all
>   >   a
>   >   > complete
>   >   > > crock if no systems put together like this even backtest 
well,
>   >   > never mind
>   >   > > forward testing.
>   >   > >
>   >   > > but every time I think about abandoning this line of
>   research, it
>   >   > seems like
>   >   > > the first thing I'd want to do with a new system would be
>   (let me
>   >   > guess),
>   >   > > test and possibly adjust it using data up to some date, 
then
>   run
>   >   > with it for
>   >   > > a while after that and see if equity growth is good. if 
it is,
>   >   I'd
>   >   > want to
>   >   > > lather, rinse and repeat with other in and out of sample
>   data, to
>   >   > make sure
>   >   > > that wasn't coincidence.
>   >   > >
>   >   > > sounds way too familiar to be a completely different 
animal.
>   >   > >
>   >   > > dave
>   >   > >   From: Fred [mailto:fctonetti@x...]
>   >   > >
>   >   > >   That IS what I was trying to say.  I suspect because 
equity
>   >   feed
>   >   > back
>   >   > >   is like looking in a rear view mirror, great for 
letting us
>   know
>   >   > >   where we were and how we could have adjusted the past to
>   make it
>   >   > >   better, but that's about it.


------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/GHeqlB/TM
---------------------------------------------------------------------~->

Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/