|
PureBytes Links
Trading Reference Links
|
MG, I just got back from Reno last week. There was plenty of sin
around but I was too busy gambling to participate in it. Oh, well.
I not quibbling with your basic premise. In fact, as far as quidelines
go they're good and probably should be put in the file section so
every newbie that asks the question can go find the guidelines without
reposting them a dozen times.
For me, the problem with both the question and the answer is the
interpretation. I thought statistics was about generating inference.
The word inference is the problem. My basic thinking has always been
that statistics does not give us answers it gives us inferred
results. Arithmetic, algebra, and sometimes calculus, gives us answers.
Newbie's want answers, not inference. Inference is messy to deal with,
answers aren't.
So if my tests produce 30 trades then I have statistically sound
results. Well that depends. If the system was tested over a long
period of time with both in sample and out of sample data and the
results from both tests were basically in agreement, then the
inference has a reasonable probability of occuring. However, if the
test period were short, the sample size too small, etc. then the
inference has a poor probability of occurance.
There's nothing absolute about the number 30. Newbie's don't want to
read any books on systems development and testing. If they did, they
would find out that different developers have different preferences.
Why? Because there is no one right answer. It all depends on your data
set, the conditions of the test, the population and more.
Some people are willing to go with an inference that has a probability
of 50% as a correct inference for the hypothesis being tested. Other
people want a 90% probability, so as you know, that means the interval
of where the next outcome lies between x and y just got bigger.
Again your guidelines are very good.
Chop off the last 20% is the out of sample test data. Use as much data
as you can. Run the tests on different market conditions to see what
happens to your system in up trends and down trends, go back and
review the charts by hand.
Everyone testing data should do all of those things. Then they should
realize that after all those tests they only have an inference. The
next outcome may not be the inferred outcome. The data set could
change enough (the market) to make the tests a poor predictor of the
future. The reasons for that are many including too much curve
fitting, changes in the market overall, changes in the stock that
causes it's data set to deviate significantly from the data set
generated over the last 20 years. These changes can take place at any
time without any advanced warning.
Maybe's are what makes trading hard. The mathematics of probability
can be a cruel teacher.
If I have 30 trades in my systems tests is that statistically
sound--maybe. If I test this over 10 years is that long enough--maybe.
If I optimize only one or two variables will that avoid curve fit--maybe.
If someone follows your approach the maybe's have a bit better chance
of becoming reality. But they're still maybe's.
--- In equismetastock@xxxxxxxxxxxxxxx, mgf_za_1999 <no_reply@xxxx> wrote:
> The comment on gambling and sin was ment to bring a smile to your
> faces, if it fails to do this, please just ignore it.
>
> To return to the original topic, I've given three methods to check for
> statistical validity of parameters of a trading system
>
> 1. 30 + degrees of freedom + a bit extra
> 2. use a hold out, chop off the last 20% or so and test it on this
> 3. break the sample in bits and build it on randomly selected bits,
> then test it on the rest
>
> Of these, only #1 directly applies to the question asked, how many
trades.
>
> I want to add another technique:
>
> 4. plot and investigate the equity line
>
> This is quite revealing, try to get an upward sloping one. If it
> spikes up in say the 80s and then stays high, but does not get higher,
> run away from the system as it overfitted to some trade opportunities
> in the 80s and does not work anymore. We quantified a method to check
> for an continually upward sloping line which is another story.
>
> You can do 4 along with all the rest, in fact, you can combine all of
> these.
>
> Superfragalist, you obviously don't like these methods, but I am not
> entirely clear why not, apart from the fact that #1 will not like a
> 9-trades-in-20-years strategy. Maybe you can shed some light by
> giving us a suggestion of :
>
> > > how many trades the evaluation needs to be
> > > sound
> > > statistically?
>
> Regards
> MG Ferreira
> TsaTsa EOD Programmer and trading model builder
> http://www.ferra4models.com
> http://fun.ferra4models.com
>
> --- In equismetastock@xxxxxxxxxxxxxxx, "metastkuser"
> <andysmith_999@xxxx> wrote:
> > 90% of traders don't know what they are doing. Much of the remaining
> > 10% are professional traders.
> >
> > 90% of gamblers don't know that they are doing. Much of the remaining
> > 10% are professional gamblers (which includes the gaming industry).
> >
> > No difference yet.
> >
> >
> > --- In equismetastock@xxxxxxxxxxxxxxx, mgf_za_1999 <no_reply@xxxx>
> wrote:
> > > If your system trades 9 times in 20 years, either give the money to
> > > some index manager, or put it in the bank. You are not trading, you
> > > are buying and holding or investing. If you add any conceivable
> > > gearing then either you will run out of margin, or pay through your
> > > ears in carry over the 20 years with just 9 trades.
> > >
> > > Anyhow, 9 trades in 20 years sounds academic to me - 30 trades plus
> > > degrees of freedom sounds practical to me.
> > >
> > > I do use such long term, 9-trades-in-20-years systems to extract the
> > > long term trend from a ticker. But I do not use that as a trading
> > > decision - just as part of the input.
> > >
> > > Yes I agree with you, trading is not investing. But I certainly
don't
> > > think trading is gambling. It is gambling if you don't know
what you
> > > are doing, probably with much worse odds than you'd get in a
gambling
> > > house. Trading is buying and selling of financial instruments
with a
> > > view to making a speculative profit while gambling is
statistical and,
> > > given the odds, a sin!
> > >
> > > Regards
> > > MG Ferreira
> > > TsaTsa EOD Programmer and trading model builder
> > > http://www.ferra4models.com
> > > http://fun.ferra4models.com
> > >
> > > --- In equismetastock@xxxxxxxxxxxxxxx, superfragalist
<no_reply@xxxx>
> > > wrote:
> > > > Your premise is from a purely mathematical view, specifically
> > > > statistical. However, the market doesn't always supply data in a
> > > > complete packages ready for statistical testing and inference.
> > > >
> > > > Suppose we have a market timing system that has made only 9
> trades in
> > > > the last 20 years and all of the trades have been highly
profitable.
> > > > Do we use the system or not? There are not enough trades to
validate
> > > > the results.
> > > >
> > > > We can wait another 40 years or so and we'll probably have
> enough data
> > > > and enough trades to make statistically meaningful inferences.
> > > >
> > > > None of this is neat, precise or absolute. And there are no
hard and
> > > > fast rules for how many trades a system needs to give good test
> > > > results. There are approaches which are better than others
like this
> > > > one by MG, but there is no one correct answer to the question.
> > > >
> > > > After many millions of systems tests and a lot of trading years
> in the
> > > > markets, no one has come with a trading system, a timing system
> or any
> > > > other system that works consistently over long periods of market
> > > history.
> > > >
> > > > Trading is not investing, it's gambling with an edge to the
> player if
> > > > the player is an expert at that game. However, the house is always
> > > > changing a little something here or there that changes the
> > > > probabilities of events just enough to change the game. It's the
> > > > players job to stay up with these changes and adapt well enough to
> > > > keep the edge on the house.
> > > >
> > > > Newbie's just don't get how long it takes and how hard it is
to get
> > > > the edge consistently and over long periods of time. A newbie
thinks
> > > > if they make money one year, they're going to be a successful
trader
> > > > every year. Call me in twenty years with your track record and
if it
> > > > measures up, I'll send you your certificate of validation.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > --- In equismetastock@xxxxxxxxxxxxxxx, mgf_za_1999 <no_reply@xxxx>
> > > wrote:
> > > > > The 30 trades is based on the central limit theorem - after
> about 30
> > > > > observations things settle down if the mean of random samples
> > follows
> > > > > a normal distribution. There are several assumptions in this
> > > > > approach, but it should give a good idea. I'd push it up a
> bit, say
> > > > > to 35 or 40. Also, you need to adjust for degrees of freedom
> if you
> > > > > do any optimisation. Suppose your system is driven by 1
> parameter,
> > > > > then you must add this to the 30. Suppose you have a big system
> > that
> > > > > uses say 10 parametrs - then you need at least 40 trades.
> > Especially
> > > > > if the system gets bigger, it needs more trades to give any
> > > > > confidence, and I will feel better if such a system produced
good
> > > > > results in 50 or more trades.
> > > > >
> > > > > Another, excellent way to test is to use a hold out sample.
> > Build the
> > > > > system on a portion of the data, say an 80% sample. Then test
> it on
> > > > > the rest and you can see if you have a winner or fools gold.
The
> > > > > *proper* way to do this is to segment the sample in say 10
> > blocks (of
> > > > > 10% of the data each). Now you choose randomly any 8 blocks,
> > optimise
> > > > > the parameters of the system on it, and test it on the
> remaining 2.
> > > > > Then you choose another 8 blocks randomly, optimise the
> system, test
> > > > > it on the remaining 2 and so on. After you've done this say 100
> > > > > times, you test the results.
> > > > >
> > > > > For this you need special software - one good example can be
> > found at
> > > > >
> > > > > http://weka.sf.net
> > > > >
> > > > > In practise, just chop off the most recent 20% and you'd get a
> good
> > > > > idea if the system will work or not.
> > > > >
> > > > > Regards
> > > > > MG Ferreira
> > > > > TsaTsa EOD Programmer and trading model builder
> > > > > http://www.ferra4models.com
> > > > > http://fun.ferra4models.com
> > > > >
> > > > >
> > > > > --- In equismetastock@xxxxxxxxxxxxxxx, "rvalue1" <rvalue1@xxxx>
> > wrote:
> > > > > > I would contend that if you generated >30 trades in the up
> > > direction
> > > > > > for a sufficiently long period 2 years or so, you would have
> > > > > > confidence that the system does well in the up direction. Same
> > for
> > > > > > down and catch the sideways as it transitions. Very
unusual to
> > > find
> > > > > > a great system up, down and sideways!! If you have one,
let me
> > > know.
> > > > > >
> > > > > > If you are waiting for 1000 trades, you must trade very often.
> > > > > >
> > > > > > --- In equismetastock@xxxxxxxxxxxxxxx, "Ed Hoopes"
> > > > > > <reefbreak_sd@xxxx> wrote:
> > > > > > > I recently attended a lecture by Keith Fitchen, the
author of
> > > > > > several
> > > > > > > successful trading systems most notably Aberration. He says
> > that
> > > > > > > statistics on more than 1000 trades must be compiled
> before the
> > > > > > > results can be considered valid.
> > > > > > >
> > > > > > > Ed Hoopes
> > > > > > >
> > > > > > > --- In equismetastock@xxxxxxxxxxxxxxx, chichungchoi
> > > <no_reply@xxxx>
> > > > > > wrote:
> > > > > > > > Does anyone know how many trades the evaluation needs
to be
> > > sound
> > > > > > > > statistically?
> > > > > > > > Thank you in advance
> > > > > > > > Eric
------------------------ Yahoo! Groups Sponsor --------------------~-->
Help Sudanese refugees rebuild their lives through GlobalGiving.
http://us.click.yahoo.com/hjNroD/EbOLAA/cosFAA/BefplB/TM
--------------------------------------------------------------------~->
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/equismetastock/
<*> To unsubscribe from this group, send an email to:
equismetastock-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|