[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [amibroker] Robustness testing & Over-optimization checklist



PureBytes Links

Trading Reference Links




<SPAN 
class=213014312-27092003>thanks, I think I made that original request, maybe 
others too.
<SPAN 
class=213014312-27092003> 
<SPAN 
class=213014312-27092003>what do you make of these:
<SPAN 
class=213014312-27092003> 
<FONT face="Courier New" color=#0000ff 
size=2>> 8. Will you test a full range of parameters.> 9. In what 
order will the parameters be tested?
<SPAN 
class=213014312-27092003>> 10. Are the parameters distributed 
properly?
 
<SPAN 
class=213014312-27092003>why would you expect profitability for all parameter 
settings? does the existence of some unprofitable settings invalidate a 
strategy? <SPAN 
class=213014312-27092003>what difference does the order in which parameters are 
tested make? what does #10 mean?
<SPAN 
class=213014312-27092003> 
<SPAN 
class=213014312-27092003>did you like the book overall? would you say it answers 
the robustness question well, beyond this checklist?
<SPAN 
class=213014312-27092003> 
<SPAN 
class=213014312-27092003>thanks,
<SPAN 
class=213014312-27092003> 
<SPAN 
class=213014312-27092003>dave
<BLOCKQUOTE 
>
  I recall someone asking about 
  over-optimization and robustness testing.  Here's an excerpt from...<SPAN 
  class=213014312-27092003><FONT face="Courier New" 
  color=#0000ff> 
  <SPAN 
  class=213014312-27092003> 
  Book: "Smarter Trading: Improving Performance in 
  Changing Markets"
  
  Chapter: Making a Strategy 
  RobustAuthor: Perry Kaufman
   
  CHECKLIST FOR ROBUST TESTING
  PART 1: Deciding What to Test<FONT 
  face=Arial size=2>
  1. Is the strategy logical?2. Can you program all the 
  rules?3. Does the strategy make sense only under certain conditions?4. 
  Take a guess as to the expected results.
  PART 2: Deciding How to Test
  5. Choose the testing tools and method.6. Do you have enough of the 
  "right" data?7. Have you Included realistic transaction costs?8. Will 
  you test a full range of parameters.9. In what order will the parameters 
  be tested?
  10. Are the parameters distributed properly?11. Have you defined the 
  evaluation criteria?12. How will the output be presented?
  PART 3: Evaluating the Results
  13. Are the calculations correct?14. Were there enough trades to be 
  "significant?"15. Does the trading system produce profits for most 
  combinations of parameters?16. Did logic changes Improve overall test 
  performance?17. How did it perform on out-of-sample data?
  PART 4: Choosing the Specific Parameters to Trade
  18. Did the last test include the most recent data?19. Did you choose 
  from an area of broad success?20. Are profits distributed relatively 
  evenly over the tested history?21. Are the profits per trade large enough 
  to absorb errors?22. Did the historic results show any large losses due to 
  price shocks?23. Have you risk-adjusted the returns to your 
  acceptable
  PART 5: Trading and Monitoring Performance
  24. Are you following the same rules that were tested?25. Are you 
  trading the same data that was tested?26. Are you monitoring the 
  difference between the system's actual entries & exits?
   Send 
  BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to 
  suggest@xxxxxxxxxxxxx-----------------------------------------Post 
  AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A 
  href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check 
  group FAQ at: <A 
  href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
  Your use of Yahoo! Groups is subject to the <A 
  href="">Yahoo! Terms of Service. 







Yahoo! Groups Sponsor


  ADVERTISEMENT 









Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html



Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.