|
PureBytes Links
Trading Reference Links
|
This is one I totally agree on ...
Prior to the 2000-X bear one had to look a long way back to find one
to include in testing. '87 was a two day event i.e. a panic not
really a long protracted bear. Prior to that you really had to go
back to 73-4 to find one. As can now be seen after the bubble, the
other mini-bears that occured in between hardly provided a sufficient
test bed for what was to come.
--- In amibroker@xxxxxxxxxxxxxxx, "quanttrader714"
<quanttrader714@xxxx> wrote:
> These are excellent questions. Quick answers for now.
>
> --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx>
> wrote:
> > some robustness issues that have been rattling around in my head
> over the
> > weekend...
> >
> > - the lite version of Robustitude #1...
>
> Don't use it. I shouldn't have mentioned it.
>
> >
> > - if one of our first tests is 3 2-year periods, that's 6 years of
> in-sample
> > testing. I'm concerned that we're running short of out-of-sample
> years.
>
> I'm going to instantly lose credibility with a lot of people with my
> answer to this, but I'd ask folks to keep an open mind until I'm
done
> posting all 5 criteria. I've read the trading and academic
literature
> on this and have done almost 2 decades of my own testing. Although
I
> still "dabble" with OOS testing (with data that exists now, e.g.,
past
> data) once something meets the robustness criteria the only OOS
> testing I'm interested in is on future data (that doesn't exist
yet).
> Not to mention that some of the criteria use all of the data. You
are
> of course free to modify my approach by saving some data in reserve,
> but your simulations won't be as accurate so it's a trade-off, but
one
> I've found worthwhile. Regarding OOS testing on future data, I'm
> *very* interested in that because as you'll see, criteria 4 and
> 5 and to a lesser extent 3 are *forward looking* and that's really
> where the rubber meets the road: how performance in real time stacks
> up against estimated future performance.
>
> >
> > - I know we're aiming for robustness, meaning wide applicability
to
> future
> > results. even given that goal, how similar do we think 1990 or
> 1995's market
> > dynamics really are to today's, especially when you get to the
level
> of time
> > constants or volume levels? how applicable to today's market is
> what we
> > learn from these older tests? how would be figure that out? and
> again, the
> > more we restrict our testing to recent years, the less out-of-
sample
> time
> > there is.
>
> I like to go back to at least the 80s for criteria 3, 4 and 5.
>
> >
> > - if one system does better in bull years and another in bear, the
> one that
> > does better in reality will depend on the proportion of bull and
> bear years
> > that actually occur. when we weight bull, bear and sideways
markets
> equally,
> > are we matching their proportions in real life? what time frame
> would we
> > want to base that judgment on?
>
> The problem is you never know when *future* conditions will be bull,
> bear or sideways and in what proportion. Remember, blunt tools, and
> equal weighting is a robust way in the absence of overwhelming
> evidence to the not do so. Especially with robust systems that do
> well under varying conditions (you'll like criterion 3).
>
> >
> > dave
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Rent DVDs from home.
Over 14,500 titles. Free Shipping
& No Late Fees. Try Netflix for FREE!
http://us.click.yahoo.com/I3w.vC/hP.FAA/3jkFAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|