[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Objective functions (was RE: [amibroker] Re: Optimization -- again)



PureBytes Links

Trading Reference Links

Hi,

I thought I might throw in my 2 cents.

Vendors love optimization, because it can generate eye popping 
hypothetical profits which has no connection to real-time trading. 

I prefer a system to work without optimization. But if I have to do 
it, I would make sure that the optimization is robust in the 
following manner:

1. The sample size of data should be large enough to represent real-
time market conditions - bull, bear and sideways markets. 

2. The look-back period should be as large as possible for the same 
reasons. 

3. The testing of optimizable parameters should be on out of sample 
data using walk-forward analysis. 

4. The Central Limit Theorem says that for a sample to assume the 
characteristics of the population, the size of sample should be 
large. The minimum sample size should be around 30. But since an 
uptrend or downtrend can last for say 50 periods, I would have a 
minimum sample size of 100 periods making sure that the full market 
cycle is there (uptrend, downtrend and congestion). 

5. The optimizable parameters should be as few as possible and tested 
in a wide variety of markets.

Curve-fitting is like rolling a fair dice with 1/6 probability of 
getting any number from 1 to 6, rolling it 5 times, getting #6, 4 out 
of 5 times (80%) of time.

A lot of traders fall in the trap of curve-fitting without being 
aware of it. So when designing a system, it is important to keep your 
guard up as far as curve-fitting is concerned.

Regards,

Pal
--- In amibroker@xxxxxxxxxxxxxxx, "Gary A. Serkhoshian" 
<serkhoshian777@xxxx> wrote:
> Fred,
>  
> Could you narrow-down your idea of a reasonable sample size for 
backtests.  You've been hinting at rather sizeable backtesting 
periods, but would like to put some numbers to it.  Also wonder if 
you use # of trades as a guide versus period of time for backtesting 
period.
>  
> Thanks,
> Gary
> 
> Fred <fctonetti@xxxx> wrote:
> There are a lot of questions and provacative statements in your 
post, 
> only one of which from my perspective needs an answer/response.
> 
> Market behavior will continually change after that ...  
> 
> Change ? from what ? into what ? I guess this is the part I don't 
> follow.  To me there is nothing new in market behavior now that 
> didn't exist last month, last year, last decade, last century, but 
> clearly those that take a short sighted view of history and the 
> market action that made up that history will clearly never see it.  
> It's a forest and trees thing ... 
> 
> --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx> 
> wrote:
> > I'm not trying to be argumentative, honest (:-)... I'm more than 
a 
> little
> > sick of saying the same thing over and over, but I  j u s t   d o 
> n ' t   g
> > e t   i t .
> > 
> > ------------------------------
> > 
> > I fail to see the huge difference in principle between equity 
> feedback and
> > backtesting.
> > 
> > let's start by assuming that backtesting performance of a system 
> and its
> > parameters over some period of past data tells you something 
about 
> its
> > future performance. it's not a perfect predictor, but it's the 
best 
> evidence
> > we have. does this seem like a reasonable starting point? what 
> alternative
> > is there?
> > 
> > if that's true, why is it better to do it only once? what 
> justification is
> > there for picking one examination period over another? clearly 
> market
> > behavior will change continually after that. don't we need a way 
of 
> working
> > that looks at what's been happening and evolves our response?
> > 
> > sounds like we examine performance up to some point and adjust, 
> trade with
> > the best-choice system and parameters for a while, then examine 
and 
> adjust
> > again later. make sense? what alternative is there?
> > 
> > so then, how often do we re-examine performance history? to put it
> > differently, how long do we ignore any changes in market dynamics 
> that may
> > or may not have occurred? why would intermittently refusing to 
look 
> and
> > respond improve system performance or reliability?
> > 
> > if that needs to be done, why not have the system itself do it, 
as 
> part of
> > its inherent operation? why is it better for us as an outside 
agent 
> to
> > periodically run some separate tests, reach into the internals of 
> the
> > system, and change stuff?
> > 
> > or should we just continue with the system and parameters we 
choose 
> at the
> > beginning? are they somehow more valid than what we'd choose 
later, 
> using
> > the same backtesting methods, but on a different date range of 
data?
> > 
> > ------------------------------
> > 
> > I realize that even if it seems to make sense logically, this all 
a 
> complete
> > crock if no systems put together like this even backtest well, 
> never mind
> > forward testing.
> > 
> > but every time I think about abandoning this line of research, it 
> seems like
> > the first thing I'd want to do with a new system would be (let me 
> guess),
> > test and possibly adjust it using data up to some date, then run 
> with it for
> > a while after that and see if equity growth is good. if it is, 
I'd 
> want to
> > lather, rinse and repeat with other in and out of sample data, to 
> make sure
> > that wasn't coincidence.
> > 
> > sounds way too familiar to be a completely different animal.
> > 
> > dave
> >   From: Fred [mailto:fctonetti@x...]
> > 
> >   That IS what I was trying to say.  I suspect because equity 
feed 
> back
> >   is like looking in a rear view mirror, great for letting us know
> >   where we were and how we could have adjusted the past to make it
> >   better, but that's about it.
> 
> 
> Yahoo! Groups SponsorADVERTISEMENT
> 
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at: 
http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
> 
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of 
Service. 
> 
> 
> ---------------------------------
> Do you Yahoo!?
> The New Yahoo! Shopping - with improved product search


------------------------ Yahoo! Groups Sponsor ---------------------~-->
Rent DVDs Online - Over 14,500 titles.
No Late Fees & Free Shipping.
Try Netflix for FREE!
http://us.click.yahoo.com/Tq9otC/XP.FAA/3jkFAA/GHeqlB/TM
---------------------------------------------------------------------~->

Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/