[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Objective functions (was RE: [amibroker] Re: Optimization -- again)



PureBytes Links

Trading Reference Links




Pal,
 
No sense arguing with Mr. "Blood and 
Guts".  He also said: "Untutored courage is useless in the face of 
educated bullets." 
 
Take care,
 
Steve
<BLOCKQUOTE 
>
  ----- Original Message ----- 
  <DIV 
  >From: 
  palsanand 
  
  To: <A title=amibroker@xxxxxxxxxxxxxxx 
  href="">amibroker@xxxxxxxxxxxxxxx 
  Sent: Monday, October 20, 2003 11:06 
  AM
  Subject: Objective functions (was RE: 
  [amibroker] Re: Optimization -- again)
  Steve,I don't use this Aberration system, so I 
  can't comment on it.  I do know that Futures Truth tests a lot of 
  systems which are essentially curve-fitted...I compare all systems 
  with my primary trading system which works for all markets (Stocks, 
  Futures and FOREX) and has been in R&D for over 45 years.  It is 
  also the most accurate T-Bond trading system ever developed...  I 
  have collected/developed over 140 systems but I'm yet to find another 
  system other than my primary trading system with which I can trade with 
  absolute confidence and even convert my losing positions into 
  profit...  The rest of the 139 or so systems have been reduced to 
  simple verification systems....George Patton once said that a Warriors 
  greatest asset is his self-confidence...Regards,Pal--- 
  In amibroker@xxxxxxxxxxxxxxx, "CedarCreekTrading" <kernish@xxxx> 
  wrote:> Pal,> > For a system to be valid, it must 
  work on all numbers tested, not just those with certain names and not 
  others > with different names.> > Try telling Keith 
  Fitchen that (the author of Aberration, the most successful mechanical 
  system ever sold ... check out Futures Truth).  First, Keith will 
  tell you that his wildly successful approach DOES NOT work on 
  equities.  Second, Keith will tell you that he only trades a basket 
  of six commodities.  I believe both these principles are directly 
  contrary to your opinions about optimizing and the selection of issues to 
  be traded.> > If a system works on Bonds and not on Beans, this 
  system is curve fitted over a specific set of data (Bonds) and it loses 
  all statistical validity.> > Wrong, wrong, wrong.  If I 
  have an approach that has worked on Bonds for ten years and it doesn't 
  work on beans...BFD.  Should I abandon a robust approach to trading 
  Bonds...because I can't make "beans" on Beans?   > 
  > Beware of drinking other people's bath water and whatever you do, 
  don't drink the Kool Aid .   > > Take 
  care,> > Steve> > >   ----- 
  Original Message ----- >   From: palsanand 
  >   To: amibroker@xxxxxxxxxxxxxxx >   Sent: 
  Monday, October 20, 2003 9:43 AM>   Subject: Objective 
  functions (was RE: [amibroker] Re: Optimization -- again)> > 
  >   Hi,> >   In my mind, curve 
  fitting means either using different systems for >   
  different markets, or using different parameters of the same system 
  >   for different markets, and this is not valid technical 
  analysis.> >   Historical testing via computer means 
  feeding a set of numbers (open, >   ow, close prices), 
  and receiving back an output set of rules that >   hopefully 
  will make money trading. The numbers themselves do not have 
  >   names, and the computer doesn't recognize the difference 
  >   between 'Beans' or 'Bonds'. For a system to be valid, it 
  must work on >   all numbers tested, not just those with 
  certain names and not others >   with different 
  names.> >   If a system works on Bonds and not on 
  Beans, this system is curve >   fitted over a specific set of 
  data (Bonds) and it loses all >   statistical validity. To 
  believe it will work in the future as it has >   worked 
  in the past is very dangerous.> >   Also, different 
  markets do not have different personalities. Again, >   
  they are reduced to just being a set of numbers or a bunch of 
  >   algorithms. If a channel breakout (or any other) method 
  is >   successful, then the same parameter must be used for 
  all the markets, >   for the same reasons as above. You 
  cannot use a 20-day channel in >   Silver and a 40-day 
  channel in Corn, this also falls under the crime >   of 
  curve fitting.> >   I therefore take exception to any 
  system, that either only trades one >   specific market 
  or group of markets, or trades different markets >   using 
  different parameters or rules of the same system. All this 
  >   proves is what has worked best in the past, and this will 
  usually not >   continue to work in the future, as there 
  is no correlation under this >   scenario.> 
  >   This is not specifically written to condemn vendors. This 
  is a >   clarification of my definitions of 'optimizing' and 
  'curve fitting', >   and a warning as to what types of 
  trading systems may be valid and >   what to stay away 
  from.> >   Regards,> >   
  Pal>   --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" 
  <dmerrill@xxxx> >   wrote:>   > 
  thanks, I'll check it out if I can find it.>   > 
  >   > I'm sure I'm ignorant, but how logic or sound 
  trading principles >   can be used>   
  > to set an MA period (for instance) without examination of past 
  >   history>   > escapes me. as does the 
  distinction between using past history or>   > 
  'experience' to do that and optimization. as does the justification 
  >   for>   > seeing optimizations from 
  one point in time as somehow blessed >   above 
  all>   > others.>   > 
  >   > dave>   >   I would 
  have to refer you to an article published by Futures >   
  Magazine>   >   concerning optimization and its 
  research value in November 20?? by>   >   
  Kent Calhoun.>   > >   >   
  Possibly the only way to do it correctly, is to first arrive at a 
  >   set>   >   of parameters 
  and algorithm based on logic, experience, or sound>   
  >   trading principals that won't be subject to change. Then do a 
  walk>   >   forward with no attempt to 
  improve results via optimization.>   > 
  >   >   Regards,>   > 
  >   >   Pal>   > 
  >   > >   >   --- In 
  amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" 
  <dmerrill@xxxx>>   >   
  wrote:>   >   > Pal, couple 
  questions/comments.>   >   
  >>   >   > - are you saying that 30 
  "occurrences" in any system produces >   
  95%>   >   accuracy?>   
  >   > 30 trades? regardless of the market or trading system 
  rules or >   time>   >   
  frame?>   >   > what's the basis for saying 
  this?>   >   >>   
  >   > - could you explain "select stable parameters with an 
  equity >   shift>   >   
  less than>   >   > the parameter shift after 
  equity spikes have been eliminated"? I>   
  >   don't>   >   > understand 
  what you mean.>   >   >>   
  >   > - just fyi, your last paragraph seems to be trying to 
  convince >   me>   >   
  that>   >   > optimizing is good, probably in 
  response to my asking "if you>   >   prefer 
  not to>   >   > optimize parameters, how do 
  you set them?". I asked that only>   >   
  because you>   >   > said, "I prefer a system 
  to work without optimization", which I>   
  >   thought was>   >   > a nice 
  goal, but one I don't understand how to achieve. seems 
  >   that>   >   you 
  don't>   >   > actually intend to avoid 
  optimization either, since you then>   >   
  discuss how you>   >   > do 
  it.>   >   >>   
  >   > dave>   >   
  >   There is a correct method to optimize any system that 
  is>   >   >   statistically valid, 
  30 occurrences with 95% accuracy.>   >   
  >>   >   >   The key to 
  optimization is to select stable parameters with an>   
  >   equity>   >   >   
  shift less than the parameter shift after equity spikes have 
  >   been>   >   
  >   eliminated. This process creates stability for optimal 
  >   parameters>   >   
  >   shifts within the four technical market phases. Parameter 
  >   shift is>   >   
  >   always geometric, but equity shift decline relative to 
  >   unstable>   >   
  >   parameter selection is usually 
  exponential.>   >   >>   
  >   >   All systems are optimized to some degree. As 
  soon as a trader>   >   
  chooses>   >   >   to enter a trade 
  on the open as opposed to the high/low/close >   
  of>   >   day,>   
  >   >   he has made a decision as to how a system 
  should be traded. >   Does>   
  >   he>   >   >   know 
  the high/low/close of day entry is better than the 
  next>   >   opening>   
  >   >   for an entry? If not why not? A potential 
  28% difference in>   >   >   
  profitability exists for channel system entries between opens 
  >   and>   >   >   
  closes.>   >   >>   
  >   >   The purpose of trading is to consistently 
  make money. This is>   >   done 
  by>   >   >   having the best 
  information available. If a trader does not >   
  know>   >   the>   
  >   >   best entry for his system, what is he trying 
  to prove? That >   the>   
  >   >   system isn't optimized? To lose money 
  because a trader is>   >   ignorant 
  of>   >   >   his systen's best 
  parameters is foolish.>   >   
  >>   >   >   
  Regards,>   >   >>   
  >   >   Pal>   >   
  >   --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" 
  >   <dmerrill@xxxx>>   
  >   >   wrote:>   >   
  >   > one question pal: if you prefer not to optimize 
  parameters, >   how>   >   
  do>   >   >   you 
  set>   >   >   > them? or do you 
  have some kind of trading rules that don't >   
  have>   >   time>   
  >   >   > constants, trigger levels, etc, that 
  need to be set?>   >   >   
  >>   >   >   > 
  dave>   >   >   
  >>   >   >   >   I 
  thought I might throw in my 2 cents.>   >   
  >   >>   >   >   
  >   Vendors love optimization, because it can generate eye 
  >   popping>   >   
  >   >   hypothetical profits which has no connection 
  to real-time>   >   
  trading.>   >   >   
  >>   >   >   >   I 
  prefer a system to work without optimization. But if I 
  >   have>   >   
  to>   >   >   
  do>   >   >   >   it, 
  I would make sure that the optimization is robust in >   
  the>   >   >   >   
  following manner:>   >   >   
  >>   >   >   >   1. 
  The sample size of data should be large enough to >   
  represent>   >   >   
  real->   >   >   >   
  time market conditions - bull, bear and sideways 
  markets.>   >   >   
  >>   >   >   >   2. 
  The look-back period should be as large as possible >   
  for the>   >   >   
  same>   >   >   >   
  reasons.>   >   >   
  >>   >   >   >   3. 
  The testing of optimizable parameters should be on out 
  >   of>   >   >   
  sample>   >   >   >   
  data using walk-forward analysis.>   >   
  >   >>   >   >   
  >   4. The Central Limit Theorem says that for a sample to 
  >   assume>   >   
  the>   >   >   >   
  characteristics of the population, the size of sample >   
  should>   >   be>   
  >   >   >   large. The minimum sample 
  size should be around 30. But >   
  since>   >   an>   
  >   >   >   uptrend or downtrend can 
  last for say 50 periods, I would>   >   have 
  a>   >   >   >   
  minimum sample size of 100 periods making sure that the 
  >   full>   >   
  >   market>   >   >   
  >   cycle is there (uptrend, downtrend and 
  congestion).>   >   >   
  >>   >   >   >   5. 
  The optimizable parameters should be as few as >   possible 
  and>   >   >   
  tested>   >   >   >   
  in a wide variety of markets.>   >   
  >   >>   >   >   
  >   Curve-fitting is like rolling a fair dice with 
  1/6>   >   probability of>   
  >   >   >   getting any number from 1 
  to 6, rolling it 5 times, >   getting>   
  >   #6, 4>   >   >   
  out>   >   >   >   of 
  5 times (80%) of time.>   >   >   
  >>   >   >   >   A 
  lot of traders fall in the trap of curve-fitting 
  without>   >   being>   
  >   >   >   aware of it. So when 
  designing a system, it is important >   
  to>   >   keep>   
  >   >   your>   >   
  >   >   guard up as far as curve-fitting is 
  concerned.>   >   >   
  >>   >   >   >   
  Regards,>   >   >   
  >>   >   >   >   
  Pal>   >   >   >   --- 
  In amibroker@xxxxxxxxxxxxxxx, "Gary A. Serkhoshian">   
  >   >   >   <serkhoshian777@xxxx> 
  wrote:>   >   >   >   
  > Fred,>   >   >   
  >   >>   >   >   
  >   > Could you narrow-down your idea of a reasonable 
  sample >   size>   >   
  for>   >   >   >   
  backtests.  You've been hinting at rather sizeable >   
  backtesting>   >   >   
  >   periods, but would like to put some numbers to it.  
  Also>   >   wonder if>   
  >   >   >   you use # of trades as a 
  guide versus period of time for>   >   
  >   backtesting>   >   
  >   >   period.>   >   
  >   >   >>   >   
  >   >   > Thanks,>   
  >   >   >   > 
  Gary>   >   >   >   
  >>   >   >   >   
  > Fred <fctonetti@xxxx> wrote:>   >   
  >   >   > There are a lot of questions and 
  provacative statements >   in>   
  >   your>   >   >   
  >   post,>   >   >   
  >   > only one of which from my perspective needs 
  an>   >   answer/response.>   
  >   >   >   >>   
  >   >   >   > Market behavior will 
  continually change after that ...>   >   
  >   >   >>   >   
  >   >   > Change ? from what ? into what ? I 
  guess this is the >   part I>   
  >   >   don't>   >   
  >   >   > follow.  To me there is nothing 
  new in market behavior >   now>   
  >   that>   >   >   
  >   > didn't exist last month, last year, last decade, 
  last>   >   century,>   
  >   >   but>   >   
  >   >   > clearly those that take a short sighted 
  view of history >   and>   
  >   the>   >   >   
  >   > market action that made up that history will clearly 
  >   never>   >   
  see>   >   >   
  it.>   >   >   >   
  > It's a forest and trees thing ...>   >   
  >   >   >>   >   
  >   >   > --- In amibroker@xxxxxxxxxxxxxxx, "Dave 
  Merrill">   >   
  <dmerrill@xxxx>>   >   >   
  >   > wrote:>   >   
  >   >   > > I'm not trying to be 
  argumentative, honest (:-)... I'm>   >   
  more>   >   >   
  than>   >   >   >   
  a>   >   >   >   > 
  little>   >   >   >   
  > > sick of saying the same thing over and over, but I  j 
  >   u s>   >   
  t>   >   >   d 
  o>   >   >   >   > 
  n ' t   g>   >   >   
  >   > > e t   i t .>   
  >   >   >   > 
  >>   >   >   >   
  > > ------------------------------>   >   
  >   >   > >>   
  >   >   >   > > I fail to see the 
  huge difference in principle between>   >   
  equity>   >   >   >   
  > feedback and>   >   >   
  >   > > backtesting.>   >   
  >   >   > >>   
  >   >   >   > > let's start by 
  assuming that backtesting performance >   of 
  a>   >   >   
  system>   >   >   >   
  > and its>   >   >   
  >   > > parameters over some period of past data tells 
  you>   >   something>   
  >   >   >   about>   
  >   >   >   > 
  its>   >   >   >   
  > > future performance. it's not a perfect predictor, 
  but>   >   it's the>   
  >   >   >   best>   
  >   >   >   > 
  evidence>   >   >   
  >   > > we have. does this seem like a reasonable 
  starting >   point?>   >   
  what>   >   >   >   
  > alternative>   >   >   
  >   > > is there?>   >   
  >   >   > >>   
  >   >   >   > > if that's true, 
  why is it better to do it only once? >   
  what>   >   >   >   
  > justification is>   >   >   
  >   > > there for picking one examination period over 
  another?>   >   clearly>   
  >   >   >   > 
  market>   >   >   >   
  > > behavior will change continually after that. don't 
  we>   >   need a>   
  >   >   way>   >   
  >   >   of>   >   
  >   >   > working>   
  >   >   >   > > that looks at 
  what's been happening and evolves our>   >   
  response?>   >   >   
  >   > >>   >   
  >   >   > > sounds like we examine performance 
  up to some point >   and>   
  >   >   adjust,>   >   
  >   >   > trade with>   
  >   >   >   > > the best-choice 
  system and parameters for a while, >   
  then>   >   >   
  examine>   >   >   >   
  and>   >   >   >   
  > adjust>   >   >   
  >   > > again later. make sense? what alternative is 
  there?>   >   >   
  >   > >>   >   
  >   >   > > so then, how often do we 
  re-examine performance >   history?>   
  >   to>   >   >   put 
  it>   >   >   >   > 
  > differently, how long do we ignore any changes in >   
  market>   >   >   
  dynamics>   >   >   
  >   > that may>   >   
  >   >   > > or may not have occurred? why 
  would intermittently>   >   refusing 
  to>   >   >   >   
  look>   >   >   >   
  > and>   >   >   
  >   > > respond improve system performance or 
  reliability?>   >   >   
  >   > >>   >   
  >   >   > > if that needs to be done, why not 
  have the system >   itself>   
  >   do>   >   >   
  it,>   >   >   >   
  as>   >   >   >   > 
  part of>   >   >   >   
  > > its inherent operation? why is it better for us as 
  an>   >   outside>   
  >   >   >   agent>   
  >   >   >   > to>   
  >   >   >   > > periodically run 
  some separate tests, reach into the>   >   
  >   internals of>   >   
  >   >   > the>   
  >   >   >   > > system, and 
  change stuff?>   >   >   
  >   > >>   >   
  >   >   > > or should we just continue with 
  the system and >   parameters>   
  >   we>   >   >   
  >   choose>   >   >   
  >   > at the>   >   
  >   >   > > beginning? are they somehow more 
  valid than what we'd>   >   
  choose>   >   >   >   
  later,>   >   >   >   
  > using>   >   >   
  >   > > the same backtesting methods, but on a different 
  date>   >   range of>   
  >   >   >   data?>   
  >   >   >   > 
  >>   >   >   >   
  > > ------------------------------>   >   
  >   >   > >>   
  >   >   >   > > I realize that 
  even if it seems to make sense >   
  logically,>   >   this>   
  >   >   all>   >   
  >   >   a>   >   
  >   >   > complete>   
  >   >   >   > > crock if no 
  systems put together like this even >   
  backtest>   >   well,>   
  >   >   >   > never 
  mind>   >   >   >   
  > > forward testing.>   >   
  >   >   > >>   
  >   >   >   > > but every time I 
  think about abandoning this line of>   >   
  >   research, it>   >   
  >   >   > seems like>   
  >   >   >   > > the first thing 
  I'd want to do with a new system >   would 
  be>   >   >   (let 
  me>   >   >   >   > 
  guess),>   >   >   >   
  > > test and possibly adjust it using data up to some 
  >   date,>   >   
  then>   >   >   
  run>   >   >   >   
  > with it for>   >   >   
  >   > > a while after that and see if equity growth is 
  good. >   if>   >   it 
  is,>   >   >   >   
  I'd>   >   >   >   
  > want to>   >   >   
  >   > > lather, rinse and repeat with other in and out of 
  >   sample>   >   
  >   data, to>   >   
  >   >   > make sure>   
  >   >   >   > > that wasn't 
  coincidence.>   >   >   
  >   > >>   >   
  >   >   > > sounds way too familiar to be a 
  completely different>   >   
  animal.>   >   >   >   
  > >>   >   >   
  >   > > dave>   >   
  >   >   > >   From: Fred 
  [mailto:fctonetti@xxxx]>   >   >   
  >   > >>   >   
  >   >   > >   That IS what I was 
  trying to say.  I suspect because>   
  >   equity>   >   >   
  >   feed>   >   >   
  >   > back>   >   
  >   >   > >   is like looking in a 
  rear view mirror, great for>   >   letting 
  us>   >   >   
  know>   >   >   >   
  > >   where we were and how we could have adjusted the 
  >   past to>   >   
  >   make it>   >   >   
  >   > >   better, but that's about 
  it.>   > >   > >   
  >         Yahoo! Groups 
  Sponsor>   > >   > 
  >   > >   >   Send BUG 
  REPORTS to bugs@xxxx>   >   Send SUGGESTIONS to 
  suggest@xxxx>   >   
  ----------------------------------------->   >   
  Post AmiQuote-related messages ONLY to: 
  amiquote@xxxxxxxxxxxxxxx>   >   (Web page: 
  <A 
  href="">http://groups.yahoo.com/group/amiquote/messages/)>   
  >   
  -------------------------------------------->   
  >   Check group FAQ at:>   > <A 
  href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html>   
  > >   >   Your use of Yahoo! Groups is 
  subject to the Yahoo! Terms of >   Service.> > 
  >         Yahoo! Groups Sponsor 
  >               
  ADVERTISEMENT>              
  >        
  >        > 
  >   Send BUG REPORTS to bugs@xxxx>   Send 
  SUGGESTIONS to suggest@xxxx>   
  ----------------------------------------->   Post 
  AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
  >   (Web page: <A 
  href="">http://groups.yahoo.com/group/amiquote/messages/)>   
  -------------------------------------------->   Check group 
  FAQ at: <A 
  href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
  > >   Your use of Yahoo! Groups is subject to the 
  Yahoo! Terms of Service.Send 
  BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to 
  suggest@xxxxxxxxxxxxx-----------------------------------------Post 
  AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A 
  href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check 
  group FAQ at: <A 
  href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
  Your use of Yahoo! Groups is subject to the <A 
  href="">Yahoo! Terms of Service. 







Yahoo! Groups Sponsor












Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html



Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.