|
PureBytes Links
Trading Reference Links
|
These are excellent questions. Quick answers for now.
--- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx>
wrote:
> some robustness issues that have been rattling around in my head
over the
> weekend...
>
> - the lite version of Robustitude #1...
Don't use it. I shouldn't have mentioned it.
>
> - if one of our first tests is 3 2-year periods, that's 6 years of
in-sample
> testing. I'm concerned that we're running short of out-of-sample
years.
I'm going to instantly lose credibility with a lot of people with my
answer to this, but I'd ask folks to keep an open mind until I'm done
posting all 5 criteria. I've read the trading and academic literature
on this and have done almost 2 decades of my own testing. Although I
still "dabble" with OOS testing (with data that exists now, e.g., past
data) once something meets the robustness criteria the only OOS
testing I'm interested in is on future data (that doesn't exist yet).
Not to mention that some of the criteria use all of the data. You are
of course free to modify my approach by saving some data in reserve,
but your simulations won't be as accurate so it's a trade-off, but one
I've found worthwhile. Regarding OOS testing on future data, I'm
*very* interested in that because as you'll see, criteria 4 and
5 and to a lesser extent 3 are *forward looking* and that's really
where the rubber meets the road: how performance in real time stacks
up against estimated future performance.
>
> - I know we're aiming for robustness, meaning wide applicability to
future
> results. even given that goal, how similar do we think 1990 or
1995's market
> dynamics really are to today's, especially when you get to the level
of time
> constants or volume levels? how applicable to today's market is
what we
> learn from these older tests? how would be figure that out? and
again, the
> more we restrict our testing to recent years, the less out-of-sample
time
> there is.
I like to go back to at least the 80s for criteria 3, 4 and 5.
>
> - if one system does better in bull years and another in bear, the
one that
> does better in reality will depend on the proportion of bull and
bear years
> that actually occur. when we weight bull, bear and sideways markets
equally,
> are we matching their proportions in real life? what time frame
would we
> want to base that judgment on?
The problem is you never know when *future* conditions will be bull,
bear or sideways and in what proportion. Remember, blunt tools, and
equal weighting is a robust way in the absence of overwhelming
evidence to the not do so. Especially with robust systems that do
well under varying conditions (you'll like criterion 3).
>
> dave
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Rent DVDs Online - Over 14,500 titles.
No Late Fees & Free Shipping.
Try Netflix for FREE!
http://us.click.yahoo.com/xlw.sC/XP.FAA/3jkFAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|