|
PureBytes Links
Trading Reference Links
|
Thomas and Mark,
I am finding this discussion informative on statistics.
I hope it continues. I am serious. This is important stuff.
What I have learned so far is that there is simple formula for
measuring whether or not results are significant (ie, dependable).
However, one needs a second test to determine if what appears to be
sigificant is really so.
It sounds similar to the supposed benefit of doing OOS testing.
If I only do 1 OOS test in my life (or if I only used Arthur
Merrill's chi-squared test once), then a positive result would be
very sigificant - ie, a positive result in an OOS test (or chi-
squared test) would be very strong confirmation that the system is
sound.
However if one uses the same OOS data over and over again (because
the first systems tested did poorly), even without consciously using
the OOS results to modify the original system, one will eventually
stumble across a setting that will work in both the In Sample data
and the OOS data.
So if I have used the OOS data only 1 or 2 times and I get a strong
confirmation of the system's performance I have some reason to be
confident. But if I am now testing my 1001st variation of a system
on OOS data, I am just fooling myself. That OOS data has become In
Sample.
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Rent DVDs from home.
Over 14,500 titles. Free Shipping
& No Late Fees. Try Netflix for FREE!
http://us.click.yahoo.com/I3w.vC/hP.FAA/3jkFAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|