|
PureBytes Links
Trading Reference Links
|
You are totally right when you say that you want to have a sample
size of 30 000 rather than of 30 to be more confident in the
statistical results...
In fact, with a sample size of 1000, the trust interval of the
result is -3/+3. It means that, if after observing 1000 events, you
get a result of x%, the "real" result is somewhere beteween (x-3)%
and (x+3)%, BUT there's NO WAY of having a better information. The
only means to get a more precise estimation is to increase the size
of the sample, but to get a -2/+2 trust interval, you need to use a
sample of 3000, and a sample of 10 000 to get down to -1/+1.
As you can see the size of the sample has to grow very fast just to
get a little more precision, and that's why the survey institutes
usually ask around 1000 people for their before elections polls : it
would be too expensive to provide better results than the -3/+3
interval...
And for a sample size of 30, the trust interval is just around -
18/+18 ! So if we use the example of the doctor claiming only 2%
chances of dying, it means that the truth is somewhere between 0 and
20%, and both values are equally true !!!! Knowing that, how many
people would go into surgery ? ;-)
Getting back to the system testing field, if your system (on a 30
trades test basis) wins in 58% of the trades, don't be surprised if
when you start using it for real, you loose more times that you
win : the actual system performance is somewhere between 40% and 76%
of winning trades (but unfortunately you can tell where, and so it
may be 40%)...
So, if you want to really trust a system (statistically at least),
you'll need to test it a lot !
Hope this helps,
Patrick
----- Original Message -----
From: "superfragalist" <no_reply@xxxxxxxxxxxxxxx>
To: <equismetastock@xxxxxxxxxxxxxxx>
Sent: Tuesday, September 06, 2005 8:20 PM
Subject: [EquisMetaStock Group] Statistical Significance ain't the
same is
Statistically Right
> In my last post I talked about some of the problems with
statistical
> inference. I probably should have mentioned statistical
significance
> versus statistically right.
>
> The central limit theorm and the number 30 trades was mentioned in
> response to someone asking how many trades it takes to be
> statistically sound.
>
> I don't know what the definition of statistically sound is but the
> answer that MG gave refers to statistically significant.
> Unfortunately, something being statistically significant does not
> address the question of dependibility.
>
> Statistically significant is a nummerical threshold that relates to
> sample size versus the size of the universe of data, or the
population.
>
> At the threshold of statistical significance, I would be very
> reluctant to place my money on any defined outcomes.
>
> In regard to statistics, the definition of how much is needed to
> produce a good enough answer is based on your defintion of how
good is
> good enough. That varies a lot from person to person and situation
to
> situation.
>
> If the question is about how many pulls is it going to take to
start
> my lawn mower, that's completely different issue than how much
data do
> I have to test to make sure I have a really high probability of
making
> money on the next 100 trades.
>
> If there's a 2% chance you won't survive a surgical procedure
that's a
> huge potential negative outcome. If there's only a 2% chance you'll
> lose your money, where's the casino!
>
> That's wrong headed thinking. Casino's make an enormous amount of
> money from people that think a 2% or less chance of losing is good
> odds. And a lot of people die from procedures that 98% survive.
>
> The central limit theorm doesn't understand nor care about the
> difference between how many pulls it takes to start my lawn mower
or
> the percentage of people who die from a surgery.
>
> If the central limit theorm says that a sample size of 30 is
> statistically significant and my doctor concludes that based on
having
> done the surgery 30 times, I only have a 2% chance of dying, I
might
> feel somewhat reluctant to embrace the central limit theorm in
regards
> to trusting it with my life. I may want to tell my doctor to call
me
> when there's been 30,000 of these surgeries and give me my odds.
>
> I suppose it all depends on how certain you need to be about the
> probability of an outcome before you're willing to bet on it.
>
> Like I said, the mathematics of probability can be a cruel teacher.
------------------------ Yahoo! Groups Sponsor --------------------~-->
Put more honey in your pocket. (money matters made easy).
http://us.click.yahoo.com/r7D80C/dlQLAA/cosFAA/BefplB/TM
--------------------------------------------------------------------~->
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/equismetastock/
<*> To unsubscribe from this group, send an email to:
equismetastock-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|