[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Robustness testing & Over-optimization checklist



PureBytes Links

Trading Reference Links




Hello,
 
I recall someone asking about over-optimization and 
robustness testing.  Here's an excerpt from...
 
Book: "Smarter Trading: Improving Performance in 
Changing Markets"

Chapter: Making a Strategy 
RobustAuthor: Perry Kaufman
 
CHECKLIST FOR ROBUST TESTING
PART 1: Deciding What to Test<FONT 
face=Arial size=2>
1. Is the strategy logical?2. Can you program all the rules?3. 
Does the strategy make sense only under certain conditions?4. Take a guess 
as to the expected results.
PART 2: Deciding How to Test
5. Choose the testing tools and method.6. Do you have enough of the 
"right" data?7. Have you Included realistic transaction costs?8. Will 
you test a full range of parameters.9. In what order will the parameters be 
tested?
10. Are the parameters distributed properly?11. Have you defined the 
evaluation criteria?12. How will the output be presented?
PART 3: Evaluating the Results
13. Are the calculations correct?14. Were there enough trades to be 
"significant?"15. Does the trading system produce profits for most 
combinations of parameters?16. Did logic changes Improve overall test 
performance?17. How did it perform on out-of-sample data?
PART 4: Choosing the Specific Parameters to Trade
18. Did the last test include the most recent data?19. Did you choose 
from an area of broad success?20. Are profits distributed relatively evenly 
over the tested history?21. Are the profits per trade large enough to absorb 
errors?22. Did the historic results show any large losses due to price 
shocks?23. Have you risk-adjusted the returns to your acceptable
PART 5: Trading and Monitoring Performance
24. Are you following the same rules that were tested?25. Are you 
trading the same data that was tested?26. Are you monitoring the difference 
between the system's actual entries & exits?
 






Yahoo! Groups Sponsor


  ADVERTISEMENT 









Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html



Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.