|
PureBytes Links
Trading Reference Links
|
Hi Fred,
In-sample/out-of-sample testing can be performed on Monte Carlo
Simulation, on its own. If you get a significantly different MCS
bell curve on the out-of-sample data, I would imagine this would
indicate the system does not function well on the OOS data. I
usually test MCS with OOS and also known bear and bull cycle data,
as well. This covers all bases.
I have never been able to quickly identify a system that works well
in MCS using a straight backtest first. However, every time I find a
system that works well in MCS, it always works well in the straight
backtest.
By "works well" I mean a system that is based in realistic real-
world statistics. That is, testing that mirrors the true randomness
of everyday reality.
MCS was originally developed to predict where nuclear fallout would
land, from nuclear tests in the Southwest desert of the US. The
predictions for this MUST be very accurate. MCS was specifically
developed for this kind of highly predictive accuracy. Weather
patterns are just as random as the stock markets.
OOS testing is specifically used for preventing curve fitting in
backtesting for system optimization. It's a whole different
enchilada. You can use OOS with MCS, however.
I hope this makes sense.
Thanks for reading,
rhelfer
--- In amibroker@xxxxxxxxxxxxxxx, "Fred" <ftonetti@xxx> wrote:
>
> I don't think your ideas or practices nullify anything ... At
worst
> they supplement it ...
>
> While I'm not a huge fan of MCS, the methodology does provide
> additional information over and above what a straight backtest
> provides ...
>
> It's my contention however that this additional information while
> beneficial is no substitute for out of sample testing regardless
of
> how it is performed.
> Fred
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|