|
PureBytes Links
Trading Reference Links
|
Hello Dave,
For #10, Kaufman is refering to the notion that the
majority (68%) of the range of parameters should be profitable. In a map
of parameter values & associated returns, you want to see a plateau instead
of just spikes. Check out the 3D optimization spreadsheet in the Files
section.
Also, the order of parameters is crucial because
you should test the most importanat variables first, i.e. the ones that cause
the largest changes in your results. Focus on those first and then on the
less important ones
Yes, the book is excellent, not just for the
robustness section. I haven't read his other trading system book yet
though so I don't know if there are similarities.
HB
<BLOCKQUOTE
>
----- Original Message -----
<DIV
>From:
Dave Merrill
To: <A title=amibroker@xxxxxxxxxxxxxxx
href="">amibroker@xxxxxxxxxxxxxxx
Sent: Saturday, September 27, 2003 8:53
AM
Subject: RE: [amibroker] Robustness
testing & Over-optimization checklist
<SPAN
class=213014312-27092003>thanks, I think I made that original request, maybe
others too.
<SPAN
class=213014312-27092003>
<SPAN
class=213014312-27092003>what do you make of these:
<SPAN
class=213014312-27092003>g
gggggggggggggggggggggg
<FONT face="Courier New"
color=#0000ff size=2>> 8. Will you test a full range of parameters.>
9. In what order will the parameters be tested?
<SPAN
class=213014312-27092003>> 10. Are the parameters distributed
properly?
<SPAN
class=213014312-27092003>why would you expect profitability for all parameter
settings? does the existence of some unprofitable settings invalidate a
strategy? <SPAN
class=213014312-27092003>what difference does the order in which parameters
are tested make? what does #10 mean?
<SPAN
class=213014312-27092003>
<SPAN
class=213014312-27092003>did you like the book overall? would you say it
answers the robustness question well, beyond this
checklist?
<SPAN
class=213014312-27092003>
<SPAN
class=213014312-27092003>thanks,
<SPAN
class=213014312-27092003>
<SPAN
class=213014312-27092003>dave
<BLOCKQUOTE
>
I recall someone asking about
over-optimization and robustness testing. Here's an excerpt
from...<FONT face="Courier New"
color=#0000ff>
<SPAN
class=213014312-27092003>
Book: "Smarter Trading: Improving Performance
in Changing Markets"
Chapter: Making a Strategy
RobustAuthor: Perry Kaufman
CHECKLIST FOR ROBUST TESTING
PART 1: Deciding What to Test<FONT
face=Arial size=2>
1. Is the strategy logical?2. Can you program all the
rules?3. Does the strategy make sense only under certain
conditions?4. Take a guess as to the expected results.
PART 2: Deciding How to Test
5. Choose the testing tools and method.6. Do you have enough of the
"right" data?7. Have you Included realistic transaction costs?8.
Will you test a full range of parameters.9. In what order will the
parameters be tested?
10. Are the parameters distributed properly?11. Have you defined
the evaluation criteria?12. How will the output be presented?
PART 3: Evaluating the Results
13. Are the calculations correct?14. Were there enough trades to be
"significant?"15. Does the trading system produce profits for most
combinations of parameters?16. Did logic changes Improve overall test
performance?17. How did it perform on out-of-sample data?
PART 4: Choosing the Specific Parameters to Trade
18. Did the last test include the most recent data?19. Did you
choose from an area of broad success?20. Are profits distributed
relatively evenly over the tested history?21. Are the profits per trade
large enough to absorb errors?22. Did the historic results show any
large losses due to price shocks?23. Have you risk-adjusted the returns
to your acceptable
PART 5: Trading and Monitoring Performance
24. Are you following the same rules that were tested?25. Are you
trading the same data that was tested?26. Are you monitoring the
difference between the system's actual entries & exits?
Send BUG REPORTS to
bugs@xxxxxxxxxxxxxSend SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page:
<A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Send
BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Yahoo! Groups Sponsor
ADVERTISEMENT
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
|