|
PureBytes Links
Trading Reference Links
|
Group,
This was the Denver Trading Group presentation a
few weeks ago. It was given by my Fort Collins buddy: Dave
Chamness. Tell me what you think.
Take care,
Steve
<BLOCKQUOTE
>
----- Original Message -----
<DIV
>From:
HB
To: <A title=amibroker@xxxxxxxxxxxxxxx
href="">amibroker@xxxxxxxxxxxxxxx
Sent: Saturday, September 27, 2003 12:28
PM
Subject: Re: [amibroker] Robustness
testing & Over-optimization checklist
Hello Dave,
For #10, Kaufman is refering to the notion that
the majority (68%) of the range of parameters should be profitable. In a
map of parameter values & associated returns, you want to see a plateau
instead of just spikes. Check out the 3D optimization spreadsheet in the
Files section.
Also, the order of parameters is crucial because
you should test the most importanat variables first, i.e. the ones that cause
the largest changes in your results. Focus on those first and then on
the less important ones
Yes, the book is excellent, not just for the
robustness section. I haven't read his other trading system book yet
though so I don't know if there are similarities.
HB
<BLOCKQUOTE
>
----- Original Message -----
<DIV
>From:
Dave Merrill
To: <A title=amibroker@xxxxxxxxxxxxxxx
href="">amibroker@xxxxxxxxxxxxxxx
Sent: Saturday, September 27, 2003 8:53
AM
Subject: RE: [amibroker] Robustness
testing & Over-optimization checklist
<SPAN
class=213014312-27092003>thanks, I think I made that original request, maybe
others too.
<SPAN
class=213014312-27092003>
<SPAN
class=213014312-27092003>what do you make of these:
<SPAN
class=213014312-27092003>g
gggggggggggggggggggggg
<FONT face="Courier New"
color=#0000ff size=2>> 8. Will you test a full range of
parameters.> 9. In what order will the parameters be
tested?
<SPAN
class=213014312-27092003>> 10. Are the parameters distributed
properly?
<SPAN
class=213014312-27092003>why would you expect profitability for all
parameter settings? does the existence of some unprofitable settings
invalidate a strategy? <FONT face="Courier New" color=#0000ff
size=2>what difference does the order in
which parameters are tested make? what does #10 mean?
<SPAN
class=213014312-27092003>
<SPAN
class=213014312-27092003>did you like the book overall? would you say it
answers the robustness question well, beyond this
checklist?
<SPAN
class=213014312-27092003>
<SPAN
class=213014312-27092003>thanks,
<SPAN
class=213014312-27092003>
<SPAN
class=213014312-27092003>dave
<BLOCKQUOTE
>
I recall someone asking about
over-optimization and robustness testing. Here's an excerpt
from...<FONT face="Courier New"
color=#0000ff>
<SPAN
class=213014312-27092003>
Book: "Smarter Trading: Improving Performance
in Changing Markets"
Chapter: Making a Strategy
RobustAuthor: Perry Kaufman
CHECKLIST FOR ROBUST TESTING
PART 1: Deciding What to
Test
1. Is the strategy logical?2. Can you program all the
rules?3. Does the strategy make sense only under certain
conditions?4. Take a guess as to the expected results.
PART 2: Deciding How to Test
5. Choose the testing tools and method.6. Do you have enough of
the "right" data?7. Have you Included realistic transaction
costs?8. Will you test a full range of parameters.9. In what order
will the parameters be tested?
10. Are the parameters distributed properly?11. Have you defined
the evaluation criteria?12. How will the output be
presented?
PART 3: Evaluating the Results
13. Are the calculations correct?14. Were there enough trades to
be "significant?"15. Does the trading system produce profits for most
combinations of parameters?16. Did logic changes Improve overall test
performance?17. How did it perform on out-of-sample data?
PART 4: Choosing the Specific Parameters to Trade
18. Did the last test include the most recent data?19. Did you
choose from an area of broad success?20. Are profits distributed
relatively evenly over the tested history?21. Are the profits per
trade large enough to absorb errors?22. Did the historic results show
any large losses due to price shocks?23. Have you risk-adjusted the
returns to your acceptable
PART 5: Trading and Monitoring Performance
24. Are you following the same rules that were tested?25. Are you
trading the same data that was tested?26. Are you monitoring the
difference between the system's actual entries & exits?
Send BUG REPORTS to
bugs@xxxxxxxxxxxxxSend SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page:
<A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Send BUG REPORTS to bugs@xxxxxxxxxxxxxSend
SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page:
<A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Send
BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Yahoo! Groups Sponsor
ADVERTISEMENT<img
src="" width=300 height=250 border=0>
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
Attachment:
optimize final.ppt
Description: MS-Powerpoint presentation
Attachment:
Attachment:
Description: "Description: MS-Word document"
Attachment:
Description: "OverOptimize.doc"
|