[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [amibroker] Re: Robustness testing & Over-optimization checklist



PureBytes Links

Trading Reference Links




<SPAN 
class=045283613-01102003>somewhat, yes, it makes sense. not completely sure 
how to quantify this though. the range of values constituting a plateau, 
within which 68% of values should be "good", should be how wide? not 
numerically how wide, but what's the principle? 
<SPAN 
class=045283613-01102003> 
<SPAN 
class=045283613-01102003>for example, SAR responds incredibly nonlinearly to 
parameter settings. there are some ranges of values where change makes no 
difference at all, some that are very sensitive, and some that produce 
nonsensical results. there's a recommended paramter range, but it's odd. what 
does that say about its robustness?
<SPAN 
class=045283613-01102003> 
<SPAN 
class=045283613-01102003>dave
<BLOCKQUOTE 
>The 
  68% thing is essentially the plateau thing.   You're right, in 
  the end, you will only select one parameter setting.  However, for 
  example, if your test show that a 10 period MA works amazingly, but a 
  9 and 11 period MA don't work, then choosing a 10 MA is not the best 
  way to go.  Just because it took you out of a few losses in your 
  historical backtest, it probably won't work well moving 
  forward.But if your 10 period MA is part of a plateau of "good" values 
  (plateau includes 7, 8, 9, 11, 12, 13), then it would be ok to use 
  10.  And 7, 8, 9, 11, 12, and 13 would encompass 68% of your "good" 
  values.  Make sense ?HB--- In 
  amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx> 
  wrote:> thanks for the feedback.> > I get that there 
  should be plateaus, not spikey peaks, in the yield map as> you 
  change paramter settings. what I don't get is the 68% thing.> > 
  does it mean you should restrict the range of each parameter so that 
  at> least 68% of the settings within that range are profitable? 
  typically,> people settle on one specific set of settings they hope 
  will work well for> some range of issues over som range of time. 
  how is the system more or less> robust by restricting the range of 
  parameters over which you tested to> arrive at the ones you 
  actually use?> > dave>   For #10, Kaufman is 
  refering to the notion that the majority (68%) of the> range of 
  parameters should be profitable.  In a map of parameter values 
  &> associated returns, you want to see a plateau instead of just 
  spikes.  Check> out the 3D optimization spreadsheet in the 
  Files section.> >   Also, the order of parameters is 
  crucial because you should test the most> importanat variables 
  first, i.e. the ones that cause the largest changes in> your 
  results.  Focus on those first and then on the less important 
  ones> >   Yes, the book is excellent, not just for 
  the robustness section.  I> haven't read his other trading 
  system book yet though so I don't know if> there are 
  similarities.> >   HBSend 
  BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to 
  suggest@xxxxxxxxxxxxx-----------------------------------------Post 
  AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A 
  href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check 
  group FAQ at: <A 
  href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
  Your use of Yahoo! Groups is subject to the <A 
  href="">Yahoo! Terms of Service. 







Yahoo! Groups Sponsor


  ADVERTISEMENT<img
src="" width=300 height=250 border=0>  









Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html



Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.