|
PureBytes Links
Trading Reference Links
|
Thanks, don't know where/what Reno is, but sounds like its asking for
fire and brimstone...
In terms of the maybe's, and after giving it some thought, I guess
9-trades-in-20-years is not that bad after all, especially in equity
markets. In currency or commodity markets I guess it would be
murder... Maybe...
Whoever you are (I mean this in general, not to you super) and
whatever you do, I guess you'd better get used to uncertainty and
maybes. Years ago people believed Newton, they believed that if you
had a big enough model, added enough explanatory power and kept on
increasing it beyond measure, you'd end up having *the* answer. You'd
be able to perfectly model anything, explain all the noise and so on.
Then the world was hit by quantum physics. Many things underlying
Black-Scholes come from the study of uncertainty at the atom level.
Scientists realised that, even at the level of electrons and neutrons,
there is uncertainty. It is as if the universe has uncertainty
embedded in it.
So, out went all the big models and in came the smaller models. And
the smaller, more compact, easier to handle, simplified models are
kicking dust in the faces of many of the bigger models. Maybe it is
just easier to maintain.
Regards
MG Ferreira
TsaTsa EOD Programmer and trading model builder
http://www.ferra4models.com
http://fun.ferra4models.com
--- In equismetastock@xxxxxxxxxxxxxxx, superfragalist <no_reply@xxxx>
wrote:
> MG, I just got back from Reno last week. There was plenty of sin
> around but I was too busy gambling to participate in it. Oh, well.
>
> I not quibbling with your basic premise. In fact, as far as quidelines
> go they're good and probably should be put in the file section so
> every newbie that asks the question can go find the guidelines without
> reposting them a dozen times.
>
> For me, the problem with both the question and the answer is the
> interpretation. I thought statistics was about generating inference.
> The word inference is the problem. My basic thinking has always been
> that statistics does not give us answers it gives us inferred
> results. Arithmetic, algebra, and sometimes calculus, gives us answers.
>
> Newbie's want answers, not inference. Inference is messy to deal with,
> answers aren't.
>
> So if my tests produce 30 trades then I have statistically sound
> results. Well that depends. If the system was tested over a long
> period of time with both in sample and out of sample data and the
> results from both tests were basically in agreement, then the
> inference has a reasonable probability of occuring. However, if the
> test period were short, the sample size too small, etc. then the
> inference has a poor probability of occurance.
>
> There's nothing absolute about the number 30. Newbie's don't want to
> read any books on systems development and testing. If they did, they
> would find out that different developers have different preferences.
> Why? Because there is no one right answer. It all depends on your data
> set, the conditions of the test, the population and more.
>
> Some people are willing to go with an inference that has a probability
> of 50% as a correct inference for the hypothesis being tested. Other
> people want a 90% probability, so as you know, that means the interval
> of where the next outcome lies between x and y just got bigger.
>
> Again your guidelines are very good.
>
> Chop off the last 20% is the out of sample test data. Use as much data
> as you can. Run the tests on different market conditions to see what
> happens to your system in up trends and down trends, go back and
> review the charts by hand.
>
> Everyone testing data should do all of those things. Then they should
> realize that after all those tests they only have an inference. The
> next outcome may not be the inferred outcome. The data set could
> change enough (the market) to make the tests a poor predictor of the
> future. The reasons for that are many including too much curve
> fitting, changes in the market overall, changes in the stock that
> causes it's data set to deviate significantly from the data set
> generated over the last 20 years. These changes can take place at any
> time without any advanced warning.
>
> Maybe's are what makes trading hard. The mathematics of probability
> can be a cruel teacher.
>
> If I have 30 trades in my systems tests is that statistically
> sound--maybe. If I test this over 10 years is that long enough--maybe.
> If I optimize only one or two variables will that avoid curve
fit--maybe.
>
> If someone follows your approach the maybe's have a bit better chance
> of becoming reality. But they're still maybe's.
>
>
>
>
> --- In equismetastock@xxxxxxxxxxxxxxx, mgf_za_1999 <no_reply@xxxx>
wrote:
> > The comment on gambling and sin was ment to bring a smile to your
> > faces, if it fails to do this, please just ignore it.
> >
> > To return to the original topic, I've given three methods to check for
> > statistical validity of parameters of a trading system
> >
> > 1. 30 + degrees of freedom + a bit extra
> > 2. use a hold out, chop off the last 20% or so and test it on this
> > 3. break the sample in bits and build it on randomly selected bits,
> > then test it on the rest
> >
> > Of these, only #1 directly applies to the question asked, how many
> trades.
> >
> > I want to add another technique:
> >
> > 4. plot and investigate the equity line
> >
> > This is quite revealing, try to get an upward sloping one. If it
> > spikes up in say the 80s and then stays high, but does not get higher,
> > run away from the system as it overfitted to some trade opportunities
> > in the 80s and does not work anymore. We quantified a method to check
> > for an continually upward sloping line which is another story.
> >
> > You can do 4 along with all the rest, in fact, you can combine all of
> > these.
> >
> > Superfragalist, you obviously don't like these methods, but I am not
> > entirely clear why not, apart from the fact that #1 will not like a
> > 9-trades-in-20-years strategy. Maybe you can shed some light by
> > giving us a suggestion of :
> >
> > > > how many trades the evaluation needs to be
> > > > sound
> > > > statistically?
> >
> > Regards
> > MG Ferreira
> > TsaTsa EOD Programmer and trading model builder
> > http://www.ferra4models.com
> > http://fun.ferra4models.com
> >
> > --- In equismetastock@xxxxxxxxxxxxxxx, "metastkuser"
> > <andysmith_999@xxxx> wrote:
> > > 90% of traders don't know what they are doing. Much of the remaining
> > > 10% are professional traders.
> > >
> > > 90% of gamblers don't know that they are doing. Much of the
remaining
> > > 10% are professional gamblers (which includes the gaming industry).
> > >
> > > No difference yet.
> > >
> > >
> > > --- In equismetastock@xxxxxxxxxxxxxxx, mgf_za_1999 <no_reply@xxxx>
> > wrote:
> > > > If your system trades 9 times in 20 years, either give the
money to
> > > > some index manager, or put it in the bank. You are not
trading, you
> > > > are buying and holding or investing. If you add any conceivable
> > > > gearing then either you will run out of margin, or pay through
your
> > > > ears in carry over the 20 years with just 9 trades.
> > > >
> > > > Anyhow, 9 trades in 20 years sounds academic to me - 30 trades
plus
> > > > degrees of freedom sounds practical to me.
> > > >
> > > > I do use such long term, 9-trades-in-20-years systems to
extract the
> > > > long term trend from a ticker. But I do not use that as a trading
> > > > decision - just as part of the input.
> > > >
> > > > Yes I agree with you, trading is not investing. But I certainly
> don't
> > > > think trading is gambling. It is gambling if you don't know
> what you
> > > > are doing, probably with much worse odds than you'd get in a
> gambling
> > > > house. Trading is buying and selling of financial instruments
> with a
> > > > view to making a speculative profit while gambling is
> statistical and,
> > > > given the odds, a sin!
> > > >
> > > > Regards
> > > > MG Ferreira
> > > > TsaTsa EOD Programmer and trading model builder
> > > > http://www.ferra4models.com
> > > > http://fun.ferra4models.com
> > > >
> > > > --- In equismetastock@xxxxxxxxxxxxxxx, superfragalist
> <no_reply@xxxx>
> > > > wrote:
> > > > > Your premise is from a purely mathematical view, specifically
> > > > > statistical. However, the market doesn't always supply data in a
> > > > > complete packages ready for statistical testing and inference.
> > > > >
> > > > > Suppose we have a market timing system that has made only 9
> > trades in
> > > > > the last 20 years and all of the trades have been highly
> profitable.
> > > > > Do we use the system or not? There are not enough trades to
> validate
> > > > > the results.
> > > > >
> > > > > We can wait another 40 years or so and we'll probably have
> > enough data
> > > > > and enough trades to make statistically meaningful inferences.
> > > > >
> > > > > None of this is neat, precise or absolute. And there are no
> hard and
> > > > > fast rules for how many trades a system needs to give good test
> > > > > results. There are approaches which are better than others
> like this
> > > > > one by MG, but there is no one correct answer to the question.
> > > > >
> > > > > After many millions of systems tests and a lot of trading years
> > in the
> > > > > markets, no one has come with a trading system, a timing system
> > or any
> > > > > other system that works consistently over long periods of market
> > > > history.
> > > > >
> > > > > Trading is not investing, it's gambling with an edge to the
> > player if
> > > > > the player is an expert at that game. However, the house is
always
> > > > > changing a little something here or there that changes the
> > > > > probabilities of events just enough to change the game. It's the
> > > > > players job to stay up with these changes and adapt well
enough to
> > > > > keep the edge on the house.
> > > > >
> > > > > Newbie's just don't get how long it takes and how hard it is
> to get
> > > > > the edge consistently and over long periods of time. A newbie
> thinks
> > > > > if they make money one year, they're going to be a successful
> trader
> > > > > every year. Call me in twenty years with your track record and
> if it
> > > > > measures up, I'll send you your certificate of validation.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > --- In equismetastock@xxxxxxxxxxxxxxx, mgf_za_1999
<no_reply@xxxx>
> > > > wrote:
> > > > > > The 30 trades is based on the central limit theorem - after
> > about 30
> > > > > > observations things settle down if the mean of random samples
> > > follows
> > > > > > a normal distribution. There are several assumptions in this
> > > > > > approach, but it should give a good idea. I'd push it up a
> > bit, say
> > > > > > to 35 or 40. Also, you need to adjust for degrees of freedom
> > if you
> > > > > > do any optimisation. Suppose your system is driven by 1
> > parameter,
> > > > > > then you must add this to the 30. Suppose you have a big
system
> > > that
> > > > > > uses say 10 parametrs - then you need at least 40 trades.
> > > Especially
> > > > > > if the system gets bigger, it needs more trades to give any
> > > > > > confidence, and I will feel better if such a system produced
> good
> > > > > > results in 50 or more trades.
> > > > > >
> > > > > > Another, excellent way to test is to use a hold out sample.
> > > Build the
> > > > > > system on a portion of the data, say an 80% sample. Then test
> > it on
> > > > > > the rest and you can see if you have a winner or fools gold.
> The
> > > > > > *proper* way to do this is to segment the sample in say 10
> > > blocks (of
> > > > > > 10% of the data each). Now you choose randomly any 8 blocks,
> > > optimise
> > > > > > the parameters of the system on it, and test it on the
> > remaining 2.
> > > > > > Then you choose another 8 blocks randomly, optimise the
> > system, test
> > > > > > it on the remaining 2 and so on. After you've done this
say 100
> > > > > > times, you test the results.
> > > > > >
> > > > > > For this you need special software - one good example can be
> > > found at
> > > > > >
> > > > > > http://weka.sf.net
> > > > > >
> > > > > > In practise, just chop off the most recent 20% and you'd get a
> > good
> > > > > > idea if the system will work or not.
> > > > > >
> > > > > > Regards
> > > > > > MG Ferreira
> > > > > > TsaTsa EOD Programmer and trading model builder
> > > > > > http://www.ferra4models.com
> > > > > > http://fun.ferra4models.com
> > > > > >
> > > > > >
> > > > > > --- In equismetastock@xxxxxxxxxxxxxxx, "rvalue1"
<rvalue1@xxxx>
> > > wrote:
> > > > > > > I would contend that if you generated >30 trades in the up
> > > > direction
> > > > > > > for a sufficiently long period 2 years or so, you would
have
> > > > > > > confidence that the system does well in the up
direction. Same
> > > for
> > > > > > > down and catch the sideways as it transitions. Very
> unusual to
> > > > find
> > > > > > > a great system up, down and sideways!! If you have one,
> let me
> > > > know.
> > > > > > >
> > > > > > > If you are waiting for 1000 trades, you must trade very
often.
> > > > > > >
> > > > > > > --- In equismetastock@xxxxxxxxxxxxxxx, "Ed Hoopes"
> > > > > > > <reefbreak_sd@xxxx> wrote:
> > > > > > > > I recently attended a lecture by Keith Fitchen, the
> author of
> > > > > > > several
> > > > > > > > successful trading systems most notably Aberration.
He says
> > > that
> > > > > > > > statistics on more than 1000 trades must be compiled
> > before the
> > > > > > > > results can be considered valid.
> > > > > > > >
> > > > > > > > Ed Hoopes
> > > > > > > >
> > > > > > > > --- In equismetastock@xxxxxxxxxxxxxxx, chichungchoi
> > > > <no_reply@xxxx>
> > > > > > > wrote:
> > > > > > > > > Does anyone know how many trades the evaluation needs
> to be
> > > > sound
> > > > > > > > > statistically?
> > > > > > > > > Thank you in advance
> > > > > > > > > Eric
------------------------ Yahoo! Groups Sponsor --------------------~-->
Put more honey in your pocket. (money matters made easy).
http://us.click.yahoo.com/r7D80C/dlQLAA/cosFAA/BefplB/TM
--------------------------------------------------------------------~->
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/equismetastock/
<*> To unsubscribe from this group, send an email to:
equismetastock-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|