[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [amibroker] Robustivity



PureBytes Links

Trading Reference Links




I have always wondered why 
a system must be usable over a range of markets.
If you only trade shares in 
run-of-the-mill equities why should a system be suited to other 
markets.
I also am assuming the term 
"markets" refer to something other than equity shares of companies, like 
commodities.
Perhaps someone can help me 
out with this Markets term.
 
 
Cheers,Graham<A 
href="">http://groups.msn.com/ASXShareTrading<A 
href="">http://groups.msn.com/FMSAustralia 


  
  <FONT 
  face=Tahoma size=2>-----Original Message-----From: Anthony 
  Faragasso [mailto:ajf1111@xxxxxxxx] Sent: Saturday, 1 November 2003 
  6:46 AMTo: amibroker@xxxxxxxxxxxxxxxSubject: Re: 
  [amibroker] Robustivity
  Steve,
   
  As you are probably aware....but for others on 
  the list.
   
  The first stage in the development of a trading 
  system is its design.  This involves the development of a trading idea 
  and then its implementation in some testable design. The second stage in this 
  process is the preliminary testing of the trading model.
   
  Testing comprises four main steps.
  1. Verify that all the formulas and rules are 
  being calculated as intended.
  2. Asses whether the combinations of formulas and 
  rules function as the theory would indicate.
  3. Develop a preliminary idea of 
  profitability.
  4. Develop a preliminary idea of 
  robustness.
   
  Robustness is an 
  important idea in trading model testing. The Webster's University dictionary 
  provides the following definition: Robust : Powerfully 
  built : Sturdy.
   
  The key word is " Sturdy". In other words, a 
  robust trading model is tough and long lasting. For our purposes, a robust 
  trading model may be identified by three characteristics:
  1. Profit over a wide range of 
  variables.
  2. Profit over a wide range of 
  markets.
  3. Profit over a wide range of market types and 
  conditions.
   
  In other words, a robust model will continue to 
  perform profitably when markets change. Since markets change constantly, the 
  more robust the model, the better.
  <BLOCKQUOTE 
  >
    ----- Original Message ----- 
    <DIV 
    >From: 
    <A title=kernish@xxxxxxxxxxx 
    href="">CedarCreekTrading 
    To: <A title=amibroker@xxxxxxxxxxxxxxx 
    href="">amibroker@xxxxxxxxxxxxxxx 
    Sent: Friday, October 31, 2003 4:12 
    PM
    Subject: Re: [amibroker] 
    Robustivity
    
    Dave,
     
    Quite honestly, I haven't heard any definitions 
    of robust at this forum.  Lot's of folks are willing to say:  
    "This is not robust...", but I have asked for specifics and definitions and 
    nobody is supplying the answers.  The system and how it's traded is not 
    the issue.  This is exactly why I decided to present the CMO5 on 
    ETF's.  The concept isn't new and the data is OOS.  Without any 
    tweaking or modification, the approach continues to produce a reasonable 
    equity line.  
     
    Would you like for me to comment on the 
    deviation of the equity from a chosen linear regression?  Fred 
    mentioned it would be nice to see an equity curve (and there is one on each 
    chart that I sent) and that should count for one of Mark's nine robust 
    measures.
     
    Since, I'm not bright enough to even guess at 
    nine measures of robustness (and so far, Mark hasn't shared), I must rely on 
    simple bench marks:  how does it perform over 8,000 issues, how does it 
    perform in different markets (futures, equities, ETF's), how does it perform 
    in different time periods, max eod dd's, max neg. excursions, avg. win/avg. 
    loss, expectancy, blah, blah, blah.
     
    Did I miss something or did someone actually 
    define robust?  If so, I hope it was better than the optimizing 
    thread.  Testing is necessary, optimizing is a spiraling death trap 
    (unless you name has DT as initials...then you understand the trap and play 
    the game on the edges of the black hole).  
     
    The CMO5 is a dandy indicator and the rules 
    that I apply to trigger the trades are simplistic.    Certainly, 
    anyone with decent software can better the results (well, maybe).  The 
    trick is to modify the approach ... without over-optimizing (tough 
    trick).
     
    If over the next six months, this system, with 
    it's defined triggers, trades the DIA, QQQ, and SPY with decent profits...is 
    it robust?  How many years must it continue to produce steady results 
    before it can be called robust?  If we look back ten years from now, 
    and it has performed as it has in the last four years, is that robust?  
    Must it also show the same kind of results in silver, bonds and 
    soybeans?  If it performs on EOD, data ... must it also knock em dead 
    in one minute time frames?  
     
    Here's a different view:  If a system 
    performs OOS for a period (x) of time and continues to perform into the 
    future, without adjustments...then I hereby declare that it is robust.  
    Why?  Because, just like technical analysis:  every input, that 
    can be considered, is discounted by an issue's price.  In technical 
    analysis, we discount all the information that flows in, out and around a 
    stock and only consider the price.  So, why not just judge a issue's 
    robustness by strictly pegging it to OOS results?  Period.  If 
    these ETF's continue to produce steady results for the next five years, I 
    expect someone to say:  "Well, maybe these ETF's with the 
    CMO5 performed well for the past eight years, but by my definition, 
    this is not robust". 
     
    I'm still waiting for the naysayers to define 
    robust and to post a system that can be monitored for results.  The 
    CMO5 is the default indicator that I use to demonstrate the character of 
    momentum oscillators.  Many of us have come to realize that the 
    indicator is not the most important piece of the puzzle.  The grail is 
    not a silver or gold chalice.  
     
    Robust and optimizing are difficult terms to 
    define.  Any definition can be argued.  Robust is like 
    beauty.  My wife is beautiful (my subjective opinion).  You might 
    find her unattractive.  
     
    The real silly part is that people want to 
    argue whether something is robust or not.  The rubber meets the road 
    when you trade.  Try flipping around sizeable accounts with something 
    that is not robust... money flies away.  
     
    Sorry that I haven't been able to answer your 
    question.  Why don't you direct you question to Mark.  He has a 
    nine point plan to evaluate what is or is not robust.  Is there a 
    possibility that one might over-optimize robustness?
     
    Take care,
     
    Steve
     
    <BLOCKQUOTE 
    >
      ----- Original Message ----- 
      <DIV 
      >From: 
      Dave Merrill 
      
      To: <A 
      title=amibroker@xxxxxxxxxxxxxxx 
      href="">amibroker@xxxxxxxxxxxxxxx 
      
      Sent: Friday, October 31, 2003 12:33 
      PM
      Subject: RE: [amibroker] 
      Robustivity
      
      <SPAN 
      class=093340919-31102003>thanks for the specifics, glad to have 'em, even 
      though that's not what I was trying to ask (:-)
      <SPAN 
      class=093340919-31102003> 
      <SPAN 
      class=093340919-31102003>what I'm most interested in isn't the system 
      itself, but what you did to convince yourself that it's robust. the way it 
      was posted implied that you disagreed with other ideas of 
      robust-ness/ivity floating around here, so I'm trying to find out what 
      your approach is.
      <SPAN 
      class=093340919-31102003> 
      <SPAN 
      class=093340919-31102003>dave
      <SPAN 
      class=093340919-31102003> 
      <BLOCKQUOTE 
      >
        If I am using the CMO5 with 
        triggers of 34/-34, I would go back and start a test to evaluate this 
        system and triggers.  The starting period would be whatever 
        date you pick (1990, '97, 2000, etc.).  
         
        Next, I run the test over 315 
        trading days (this period gives me results for approximately one 
        year..it takes "x" amount of periods to load the TRIX(21), which I use 
        as a trend identifier.  My approach produces about 10 to 15 round 
        turn trades a year... in each stock.  
         
        I then rank all issues by one 
        criteria:  percent return per day (while the money is in the 
        market).  If you only consider the percent per day contributions, I 
        think you will find that all other "book learned" ratios come out just 
        fine.  Numbers lie.  Would you rather trade a $100 stock that 
        returns $20 or a $20 stock that returns $10?  Percent per goes a 
        long way to normalizing the comparisons.
         
        I pick the 20 best percent per day 
        stocks and trade them for the next quarter.  At the end of the 
        quarter, I reevaluate the percentage per day contributions and reshuffle 
        the issues in play, if necessary.
         
        Symtems don't go bad, stocks and 
        commodities go bad.  Going bad is best defined by a change in the 
        pattern of supply and demand.  The cream rises to the top of 
        the list.    
         
        Is this optimizing?  Could be, 
        by some definitions.  If all the odds are even money, who would you 
        prefer to bet on:  Chicago or Kansas City?  KC is undefeated 
        and Chicago couldn't beat the local high school.  My money is on 
        KC.
         
        The stock betting setup is not 
        handicapped (like almost all games).  This is basically a even 
        money play (with subtractions for commission and 
        slippage...juice/vigorish).  If you have 9,000 issues to play, why 
        won't someone want to bet on the strongest performance?
         
        I know that the explanation might 
        be over simplified...but, the people who know me, in and out of this 
        forum, know that this is the way I do it.  I'm not crusading for 
        anything.  This works.  I've presented this simplistic 
        approach publicly to large groups and in a number of internet 
        seminars. It continues to crank out extraordinary 
        profits. 
         
        Please let me know if the 
        paragraphs help to explain the ranking.
         
        Take care,
         
        Steve
         
         
        ----- Original Message ----- 
        <BLOCKQUOTE 
        >
          <DIV 
          >From: 
          Dave 
          Merrill 
          To: <A 
          title=amibroker@xxxxxxxxxxxxxxx 
          href="">amibroker@xxxxxxxxxxxxxxx 
          
          Sent: Friday, October 31, 2003 
          9:29 AM
          Subject: RE: [amibroker] 
          Robustivity
          
          <SPAN 
          class=496400216-31102003>steve, thanks for your 
          response.
          <SPAN 
          class=496400216-31102003> 
          <SPAN 
          class=496400216-31102003>from your msg subject and the way you 
          presented this system, I thought you were offering it as an 
          example of one you had objectively evaluated and determined to 
          be robust. I was interested in how you thought "robustivity" 
          should be evaluated, since you seemed to be contrasting your 
          approach to walkforward optimization and the various other system 
          measures people were talking about.
          <SPAN 
          class=496400216-31102003> 
          <SPAN 
          class=496400216-31102003>what I'm hearing in your response below isn't 
          what I would describe as a specific method for distinguishing 
          accidentally gorgeous backtest results from robustness. you do mention 
          testing also at faster time frames, which isn't a technique that's 
          been mentioned recently. but mostly, the robustness label here seems 
          to come from your integration of various aspects of your long 
          experience with it, like your visual sense of how it behaves. am I 
          missing something?
          <SPAN 
          class=496400216-31102003> 
          <SPAN 
          class=496400216-31102003>another question: you mention issue 
          selection, the idea of looking for stocks you think will trade well 
          with a particular indicator, rather than the other way around. how do 
          you do that? by measuring raw past growth trading that indicator? 
          other measures?
          <SPAN 
          class=496400216-31102003> 
          <SPAN 
          class=496400216-31102003>thanks again,
          <SPAN 
          class=496400216-31102003> 
          <SPAN 
          class=496400216-31102003>dave
          <BLOCKQUOTE 
          >
            <FONT face="Courier New" 
            color=#0000ff size=2>just for my understanding, in what sense is 
            this system "robust"? 
            <FONT face="Courier New" 
            color=#0000ff size=2> 
            Well, 
            first, this was presented to the public in the late 90's, at a 
            series of seminars that I conducted for Equis.  Same indicator, 
            same triggers, same everything.  This robust "thing" is a tough 
            one to define.  I'll try to explain what's important to me, 
            but, it's very subjective and just one person's opinion.  
            
            <FONT face=Arial 
            size=2> 
            <FONT face="Courier New" 
            color=#0000ff size=2>is it because 
            results are similar with different similar periods and 
            thresholds?
            <FONT face=Arial 
            size=2> 
            If you 
            take this CMO5 indicator and step down in time (5, 10, 60 
            minutes), you need to widen the triggers to obtain decent 
            results.  Other than that, it trades through time-zones 
            with very good results.
            <FONT face=Arial 
            size=2> 
            <FONT face="Courier New" 
            color=#0000ff size=2>that seems unlikely, since there isn't very far 
            to go from 5 to hit 1 and 0, which I'd guess are significantly 
            different. what sort of testing led you to decide on this period and 
            threshold, and this system for that matter?
            <FONT face=Arial 
            size=2> 
            If 
            you're referring to the CMO5...I first started testing it six years 
            ago.  I've tested and eyeballed every version of CMO(x).  
            I've created a few indicators that combines different periods of the 
            CMO.  For my money, for my style, this judge of momentum trades 
            more things, more accurately than any other indicator I am aware 
            of.  As I have begged many times:  give me something 
            better...I'll use it instead of this.
            <FONT face=Arial 
            size=2> 
            <FONT face="Courier New" 
            color=#0000ff size=2>is it robust 
            because it works well on many stocks, indexes and funds over a long 
            period of time? 
            <FONT face="Courier New" 
            color=#0000ff size=2><SPAN 
            class=468263723-30102003> 
            Yes, it 
            works well on many stocks and indexes.  I don't trade funds, 
            but, some fund managers, DTG members, use versions of the CMO to aid 
            their timing.  
            <FONT face=Arial 
            size=2> 
            <SPAN 
            class=468263723-30102003><FONT face="Courier New" color=#0000ff 
            size=2>because of the concepts behind the indicator 
            itself?
            <SPAN 
            class=468263723-30102003><FONT face="Courier New" color=#0000ff 
            size=2> 
            <SPAN 
            class=468263723-30102003>I process 
            visually.  The math is beyond me.  My bottom line has 
            always been the same:  give me an indicator that is smooth, yet 
            sensitive to intermediate and major market turns.  After 
            gawking hundreds of charts, everyday, for the last six years, I'm 
            amazed at how this indicator quantifies momentum.  I like 
            versions of the Stochastic RSI and the Standard Error Oscillator, 
            but dollar for dollar, the CMO does it for 
            me.
            <SPAN 
            class=468263723-30102003><FONT face=Arial 
            size=2> 
            <SPAN 
            class=468263723-30102003><FONT face="Courier New" color=#0000ff 
            size=2>something 
            else?
            <SPAN 
            class=468263723-30102003><FONT face="Courier New" color=#0000ff 
            size=2><SPAN 
            class=468263723-30102003> 
            <SPAN 
            class=468263723-30102003><SPAN 
            class=468263723-30102003>I think there's a few other things to 
            mention.  First of all, the ETF's that I showed were chosen 
            because they represent a broad range of stocks and are popular 
            trading instruments.  Do I suggest trading these issues 
            with this system?  No way.  The CMO5 trades a lot of other 
            issues with better results than the ETF's.  I always allow the 
            issues "to pick themselves".  Trade the issues that return the 
            greatest percentages in a stable system.  
            
            <SPAN 
            class=468263723-30102003><SPAN 
            class=468263723-30102003> 
            <SPAN 
            class=468263723-30102003><SPAN 
            class=468263723-30102003>In it's stripped down version, as 
            presented, the CMO5 is an indicator that can return steady profits 
            (see equity lines) in it's rawest unoptimized form.  Is that 
            robust?  
            <SPAN 
            class=468263723-30102003><SPAN 
            class=468263723-30102003> 
            <SPAN 
            class=468263723-30102003><SPAN 
            class=468263723-30102003>Robustness and optimizing/over-optimizing 
            are fascinating and misunderstood subjects.  Over the years, 
            I've constantly simplified my approaches.  I can improve on the 
            results of the three ETF's by simply "tweaking" the trigger 
            levels.  But, will it walk forward better than the default 
            triggers of 34/-34?  At least what I presented was out of 
            sample.  
            <SPAN 
            class=468263723-30102003><SPAN 
            class=468263723-30102003> 
            <SPAN 
            class=468263723-30102003><SPAN 
            class=468263723-30102003>If an approach does a good job of 
            identifying movement of supply and demand, the approach should not 
            be expected to work on all issues.  To say a system needs 
            to work on all  issues is total crap.   To 
            say that a system sucks because it doesn't work on XYZ is another 
            large pile.  Build simple things and concentrate on issue 
            selection.
            <SPAN 
            class=468263723-30102003><SPAN 
            class=468263723-30102003> 
            <FONT face=Arial 
            size=2>Optimization leads to dark and spooky places.  Ranking 
            leads you down the yellow brick road.
            <FONT face=Arial 
            size=2> 
            Take 
            care,
            <FONT face=Arial 
            size=2> 
            <FONT face=Arial 
            size=2>Steve
            <FONT face=Arial 
            size=2> 
            <BLOCKQUOTE 
            >
              <SPAN 
              class=468263723-30102003>steve, thanks for sharing this 
              (again).
              <SPAN 
              class=468263723-30102003> 
              <SPAN 
              class=468263723-30102003> 
              <SPAN 
              class=468263723-30102003>just for my understanding, in what sense 
              is this system "robust"? 
              <SPAN 
              class=468263723-30102003> 
              <SPAN 
              class=468263723-30102003>is it because results are similar with 
              different similar periods and thresholds? that seems unlikely, 
              since there isn't very far to go from 5 to hit 1 and 0, which I'd 
              guess are significantly different. what sort of testing led you to 
              decide on this period and threshold, and this system for that 
              matter?
              <SPAN 
              class=468263723-30102003> 
              <SPAN 
              class=468263723-30102003>is it robust because it works well on 
              many stocks, indexes and funds over a long period of time? 
              
              <SPAN 
              class=468263723-30102003> 
              <SPAN 
              class=468263723-30102003>because of the concepts behind the 
              indicator itself?
              <SPAN 
              class=468263723-30102003> 
              <SPAN 
              class=468263723-30102003>something else?
              <SPAN 
              class=468263723-30102003> 
              <SPAN 
              class=468263723-30102003> 
              <SPAN 
              class=468263723-30102003>I'm not disputing the system's value, 
              which I haven't tested yet. I'm trying to understand what kind of 
              process you go through to settle on a system and 
              settings.
              <SPAN 
              class=468263723-30102003> 
              <SPAN 
              class=468263723-30102003>thanks,
              <SPAN 
              class=468263723-30102003> 
              <SPAN 
              class=468263723-30102003>dave
              <SPAN 
              class=468263723-30102003> 
              <BLOCKQUOTE 
              >
                1.  This exact system was 
                presented over a year ago at this forum
                2.  The charts are OOS (since, 
                it's been posted publicly forever)
                3.  Rules are simple:  
                Buy the opening of the next day when the CMO5 closes below -34 
                and sell when it triggers above 34.
                 
                Works on most issues (raw).  
                Works better if:  
                 
                a.  You take trades only with 
                the trend
                b.  You protect yourself from 
                large drawdowns (stop)
                c.  You conjure a profit 
                target (limit)
                d.  You put in a time 
                stop 
                 
                This is the guts of an indicator 
                and a logical systematic approach.  Whistles and bells are 
                optional (but, in my opinion necessary).  Again, if you 
                start with a pig, the prom dress doesn't make it look any 
                better.  Don't hang ornaments on a twisted Christmas 
                tree.Send 
          BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to 
          suggest@xxxxxxxxxxxxx-----------------------------------------Post 
          AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web 
          page: <A 
          href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check 
          group FAQ at: <A 
          href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
          Your use of Yahoo! Groups is subject to the <A 
          href="">Yahoo! Terms of 
          Service. Send BUG REPORTS to 
        bugs@xxxxxxxxxxxxxSend SUGGESTIONS to 
        suggest@xxxxxxxxxxxxx-----------------------------------------Post 
        AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web 
        page: <A 
        href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check 
        group FAQ at: <A 
        href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
        Your use of Yahoo! Groups is subject to the <A 
        href="">Yahoo! Terms of 
        Service. Send BUG REPORTS to 
      bugs@xxxxxxxxxxxxxSend SUGGESTIONS to 
      suggest@xxxxxxxxxxxxx-----------------------------------------Post 
      AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: 
      <A 
      href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check 
      group FAQ at: <A 
      href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
      Your use of Yahoo! Groups is subject to the <A 
      href="">Yahoo! Terms of Service. 
      
    Send BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to 
    suggest@xxxxxxxxxxxxx-----------------------------------------Post 
    AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: 
    <A 
    href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check 
    group FAQ at: <A 
    href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
    Your use of Yahoo! Groups is subject to the <A 
    href="">Yahoo! Terms of Service. 
    
  <BLOCKQUOTE 
  ><FONT 
    face="Courier New">---Outgoing mail is certified Virus 
    Free.Checked by AVG anti-virus system (<A 
    href="">http://www.grisoft.com).Version: 
    6.0.532 / Virus Database: 326 - Release Date: 
  10/27/2003Send 
  BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to 
  suggest@xxxxxxxxxxxxx-----------------------------------------Post 
  AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A 
  href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check 
  group FAQ at: <A 
  href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
  Your use of Yahoo! Groups is subject to the <A 
  href="">Yahoo! Terms of Service. 







Yahoo! Groups Sponsor


  ADVERTISEMENT 









Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html



Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.