[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [amibroker] Re: Robustivity



PureBytes Links

Trading Reference Links




Fred,
 
I was making a joke at the end of a long 
week.  I'm sure you got it.
 
Take care,
 
Steve
<BLOCKQUOTE 
>
  ----- Original Message ----- 
  <DIV 
  >From: 
  Fred 
  To: <A title=amibroker@xxxxxxxxxxxxxxx 
  href="">amibroker@xxxxxxxxxxxxxxx 
  Sent: Friday, October 31, 2003 3:24 
  PM
  Subject: [amibroker] Re: 
Robustivity
  Over-optimize robustness ?!That's an interesting 
  concept, but I think not possible due to mutual exclusivity but I suspect 
  at least someone out there won't agree.--- In <A 
  href="">amibroker@xxxxxxxxxxxxxxx, 
  "CedarCreekTrading" <kernish@x...> 
  wrote:> Dave,> > Quite honestly, I haven't heard any 
  definitions of robust at this forum.  Lot's of folks are willing to 
  say:  "This is not robust...", but I have asked for specifics and 
  definitions and nobody is supplying the answers.  The system and how 
  it's traded is not the issue.  This is exactly why I decided to 
  present the CMO5 on ETF's.  The concept isn't new and the data is 
  OOS.  Without any tweaking or modification, the approach continues to 
  produce a reasonable equity line.  > > Would you like 
  for me to comment on the deviation of the equity from a chosen linear 
  regression?  Fred mentioned it would be nice to see an equity curve 
  (and there is one on each chart that I sent) and that should count for one 
  of Mark's nine robust measures.> > Since, I'm not bright enough 
  to even guess at nine measures of robustness (and so far, Mark hasn't 
  shared), I must rely on simple bench marks:  how does it perform over 
  8,000 issues, how does it perform in different markets (futures, equities, 
  ETF's), how does it perform in different time periods, max eod dd's, max 
  neg. excursions, avg. win/avg. loss, expectancy, blah, blah, blah.> 
  > Did I miss something or did someone actually define robust?  If 
  so, I hope it was better than the optimizing thread.  Testing is 
  necessary, optimizing is a spiraling death trap (unless you name has 
  DT as initials...then you understand the trap and play the game on the 
  edges of the black hole).  > > The CMO5 is a dandy 
  indicator and the rules that I apply to trigger the trades are 
  simplistic.    Certainly, anyone with decent software can 
  better the results (well, maybe).  The trick is to modify the 
  approach ... without over-optimizing (tough trick).> > If 
  over the next six months, this system, with it's defined triggers, trades 
  the DIA, QQQ, and SPY with decent profits...is it robust?  How many 
  years must it continue to produce steady results before it can be called 
  robust?  If we look back ten years from now, and it has performed as 
  it has in the last four years, is that robust?  Must it also show the 
  same kind of results in silver, bonds and soybeans?  If it performs 
  on EOD, data ... must it also knock em dead in one minute time 
  frames?  > > Here's a different view:  If a system 
  performs OOS for a period (x) of time and continues to perform into the 
  future, without adjustments...then I hereby declare that it is 
  robust.  Why?  Because, just like technical analysis:  
  every input, that can be considered, is discounted by an issue's 
  price.  In technical analysis, we discount all the information that 
  flows in, out and around a stock and only consider the price.  So, 
  why not just judge a issue's robustness by strictly pegging it to OOS 
  results?  Period.  If these ETF's continue to produce steady 
  results for the next five years, I expect someone to say:  "Well, 
  maybe these ETF's with the CMO5 performed well for the past eight years, 
  but by my definition, this is not robust". > > I'm still 
  waiting for the naysayers to define robust and to post a system that can 
  be monitored for results.  The CMO5 is the default indicator that I 
  use to demonstrate the character of momentum oscillators.  Many of us 
  have come to realize that the indicator is not the most important piece of 
  the puzzle.  The grail is not a silver or gold chalice.  
  > > Robust and optimizing are difficult terms to define.  
  Any definition can be argued.  Robust is like beauty.  My wife 
  is beautiful (my subjective opinion).  You might find her 
  unattractive.  > > The real silly part is that people want 
  to argue whether something is robust or not.  The rubber meets the 
  road when you trade.  Try flipping around sizeable accounts with 
  something that is not robust... money flies away.  > > 
  Sorry that I haven't been able to answer your question.  Why don't 
  you direct you question to Mark.  He has a nine point plan to 
  evaluate what is or is not robust.  Is there a possibility that one 
  might over-optimize robustness?> > Take care,> 
  > Steve> >   ----- Original Message ----- 
  >   From: Dave Merrill >   To: 
  amibroker@xxxxxxxxxxxxxxx >   Sent: Friday, October 31, 2003 
  12:33 PM>   Subject: RE: [amibroker] Robustivity> 
  > >   thanks for the specifics, glad to have 'em, even 
  though that's not what I was trying to ask (:-)> 
  >   what I'm most interested in isn't the system itself, but 
  what you did to convince yourself that it's robust. the way it was posted 
  implied that you disagreed with other ideas of robust-ness/ivity 
  floating around here, so I'm trying to find out what your approach 
  is.> >   dave> >     
  If I am using the CMO5 with triggers of 34/-34, I would go back and start 
  a test to evaluate this system and triggers.  The starting period 
  would be whatever date you pick (1990, '97, 2000, etc.).  > 
  >     Next, I run the test over 315 trading days 
  (this period gives me results for approximately one year..it takes "x" 
  amount of periods to load the TRIX(21), which I use as a trend 
  identifier.  My approach produces about 10 to 15 round turn trades a 
  year... in each stock.  > >     I then 
  rank all issues by one criteria:  percent return per day (while the 
  money is in the market).  If you only consider the percent per day 
  contributions, I think you will find that all other "book learned" ratios 
  come out just fine.  Numbers lie.  Would you rather trade a $100 
  stock that returns $20 or a $20 stock that returns $10?  Percent per 
  goes a long way to normalizing the comparisons.> 
  >     I pick the 20 best percent per day stocks and 
  trade them for the next quarter.  At the end of the quarter, I 
  reevaluate the percentage per day contributions and reshuffle the issues 
  in play, if necessary.> >     Symtems 
  don't go bad, stocks and commodities go bad.  Going bad is best 
  defined by a change in the pattern of supply and demand.  The cream 
  rises to the top of the list.    > 
  >     Is this optimizing?  Could be, by some 
  definitions.  If all the odds are even money, who would you prefer to 
  bet on:  Chicago or Kansas City?  KC is undefeated and Chicago 
  couldn't beat the local high school.  My money is on KC.> 
  >     The stock betting setup is not handicapped 
  (like almost all games).  This is basically a even money play (with 
  subtractions for commission and slippage...juice/vigorish).  If you 
  have 9,000 issues to play, why won't someone want to bet on the strongest 
  performance?> >     I know that the 
  explanation might be over simplified...but, the people who know me, in and 
  out of this forum, know that this is the way I do it.  I'm not 
  crusading for anything.  This works.  I've presented this 
  simplistic approach publicly to large groups and in a number of internet 
  seminars. It continues to crank out extraordinary profits. > 
  >     Please let me know if the paragraphs help to 
  explain the ranking.> >     Take 
  care,> >     Steve> > 
  >     ----- Original Message ----- 
  >       From: Dave Merrill 
  >       To: amibroker@xxxxxxxxxxxxxxx 
  >       Sent: Friday, October 31, 2003 
  9:29 AM>       Subject: RE: [amibroker] 
  Robustivity> > >       
  steve, thanks for your response.> 
  >       from your msg subject and the way 
  you presented this system, I thought you were offering it as an example of 
  one you had objectively evaluated and determined to be robust. I was 
  interested in how you thought "robustivity" should be evaluated, since you 
  seemed to be contrasting your approach to walkforward optimization and 
  the various other system measures people were talking about.> 
  >       what I'm hearing in your response 
  below isn't what I would describe as a specific method for distinguishing 
  accidentally gorgeous backtest results from robustness. you do mention 
  testing also at faster time frames, which isn't a technique that's been 
  mentioned recently. but mostly, the robustness label here seems to 
  come from your integration of various aspects of your long experience 
  with it, like your visual sense of how it behaves. am I missing 
  something?> >       another 
  question: you mention issue selection, the idea of looking for stocks you 
  think will trade well with a particular indicator, rather than the other 
  way around. how do you do that? by measuring raw past growth trading that 
  indicator? other measures?> 
  >       thanks again,> 
  >       
  dave>         just for my 
  understanding, in what sense is this system "robust"? > 
  >         Well, first, this was 
  presented to the public in the late 90's, at a series of seminars that I 
  conducted for Equis.  Same indicator, same triggers, same 
  everything.  This robust "thing" is a tough one to define.  I'll 
  try to explain what's important to me, but, it's very subjective and just 
  one person's opinion.  > 
  >         is it because results 
  are similar with different similar periods and thresholds?> 
  >         If you take this CMO5 
  indicator and step down in time (5, 10, 60 minutes), you need to widen the 
  triggers to obtain decent results.  Other than that, it trades 
  through time-zones with very good results.> 
  >         that seems unlikely, 
  since there isn't very far to go from 5 to hit 1 and 0, which I'd guess 
  are significantly different. what sort of testing led you to decide on 
  this period and threshold, and this system for that matter?> 
  >         If you're referring 
  to the CMO5...I first started testing it six years ago.  I've tested 
  and eyeballed every version of CMO(x).  I've created a few indicators 
  that combines different periods of the CMO.  For my money, for my 
  style, this judge of momentum trades more things, more accurately than any 
  other indicator I am aware of.  As I have begged many times:  
  give me something better...I'll use it instead of this.> 
  >         is it robust because 
  it works well on many stocks, indexes and funds over a long period of 
  time? > >         Yes, 
  it works well on many stocks and indexes.  I don't trade funds, but, 
  some fund managers, DTG members, use versions of the CMO to aid their 
  timing.  > 
  >         because of the 
  concepts behind the indicator itself?> 
  >         I process 
  visually.  The math is beyond me.  My bottom line has always 
  been the same:  give me an indicator that is smooth, yet sensitive to 
  intermediate and major market turns.  After gawking hundreds of 
  charts, everyday, for the last six years, I'm amazed at how this indicator 
  quantifies momentum.  I like versions of the Stochastic RSI and the 
  Standard Error Oscillator, but dollar for dollar, the CMO does it for 
  me.> >         something 
  else?> >         I think 
  there's a few other things to mention.  First of all, the ETF's that 
  I showed were chosen because they represent a broad range of stocks and 
  are popular trading instruments.  Do I suggest trading these issues 
  with this system?  No way.  The CMO5 trades a lot of other 
  issues with better results than the ETF's.  I always allow the issues 
  "to pick themselves".  Trade the issues that return the greatest 
  percentages in a stable system.  > 
  >         In it's stripped down 
  version, as presented, the CMO5 is an indicator that can return steady 
  profits (see equity lines) in it's rawest unoptimized form.  Is that 
  robust?  > 
  >         Robustness and 
  optimizing/over-optimizing are fascinating and misunderstood 
  subjects.  Over the years, I've constantly simplified my 
  approaches.  I can improve on the results of the three ETF's by 
  simply "tweaking" the trigger levels.  But, will it walk forward 
  better than the default triggers of 34/-34?  At least what I 
  presented was out of sample.  > 
  >         If an approach does a 
  good job of identifying movement of supply and demand, the approach should 
  not be expected to work on all issues.  To say a system needs to work 
  on all  issues is total crap.   To say that a system sucks 
  because it doesn't work on XYZ is another large pile.  Build simple 
  things and concentrate on issue selection.> 
  >         Optimization leads to 
  dark and spooky places.  Ranking leads you down the yellow brick 
  road.> >         Take 
  care,> >         
  Steve> 
  >           steve, 
  thanks for sharing this (again).> > 
  >           just for 
  my understanding, in what sense is this system "robust"? > 
  >           is it 
  because results are similar with different similar periods and thresholds? 
  that seems unlikely, since there isn't very far to go from 5 to hit 1 and 
  0, which I'd guess are significantly different. what sort of testing led 
  you to decide on this period and threshold, and this system for that 
  matter?> 
  >           is it 
  robust because it works well on many stocks, indexes and funds over a long 
  period of time? > 
  >           because 
  of the concepts behind the indicator itself?> 
  >           something 
  else?> > 
  >           I'm not 
  disputing the system's value, which I haven't tested yet. I'm trying to 
  understand what kind of process you go through to settle on a system and 
  settings.> 
  >           
  thanks,> 
  >           
  dave> 
  >             
  1.  This exact system was presented over a year ago at this 
  forum>             
  2.  The charts are OOS (since, it's been posted publicly 
  forever)>             
  3.  Rules are simple:  Buy the opening of the next day when the 
  CMO5 closes below -34 and sell when it triggers above 34.> 
  >             
  Works on most issues (raw).  Works better if:  > 
  >             
  a.  You take trades only with the 
  trend>             
  b.  You protect yourself from large drawdowns 
  (stop)>             
  c.  You conjure a profit target 
  (limit)>             
  d.  You put in a time stop > 
  >             
  This is the guts of an indicator and a logical systematic approach.  
  Whistles and bells are optional (but, in my opinion necessary).  
  Again, if you start with a pig, the prom dress doesn't make it look any 
  better.  Don't hang ornaments on a twisted Christmas tree.> 
  > >       Send BUG REPORTS to 
  bugs@xxxx>       Send SUGGESTIONS to 
  suggest@xxxx>       
  ----------------------------------------->       
  Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
  >       (Web page: <A 
  href="">http://groups.yahoo.com/group/amiquote/messages/)>       
  -------------------------------------------->       
  Check group FAQ at: <A 
  href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
  > >       Your use of Yahoo! 
  Groups is subject to the Yahoo! Terms of Service. > > 
  > >     Send BUG REPORTS to 
  bugs@xxxx>     Send SUGGESTIONS to 
  suggest@xxxx>     
  ----------------------------------------->     Post 
  AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
  >     (Web page: <A 
  href="">http://groups.yahoo.com/group/amiquote/messages/)>     
  -------------------------------------------->     
  Check group FAQ at: <A 
  href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
  > >     Your use of Yahoo! Groups is subject 
  to the Yahoo! Terms of Service. > > 
  >         Yahoo! Groups Sponsor 
  >        
  >        > 
  >   Send BUG REPORTS to bugs@xxxx>   Send 
  SUGGESTIONS to suggest@xxxx>   
  ----------------------------------------->   Post 
  AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
  >   (Web page: <A 
  href="">http://groups.yahoo.com/group/amiquote/messages/)>   
  -------------------------------------------->   Check group 
  FAQ at: <A 
  href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
  > >   Your use of Yahoo! Groups is subject to the 
  Yahoo! Terms of Service.Send 
  BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to 
  suggest@xxxxxxxxxxxxx-----------------------------------------Post 
  AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A 
  href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check 
  group FAQ at: <A 
  href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
  Your use of Yahoo! Groups is subject to the <A 
  href="">Yahoo! Terms of Service. 







Yahoo! Groups Sponsor


  ADVERTISEMENT 









Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html



Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.