|
PureBytes Links
Trading Reference Links
|
Hello Sebastian,
I thought you would be interested in my prognostications on the
subject.
I like your first analogy.
That is precisely what I am pointing at and wondering what others
are thinking about failure rates.
I am only primarily interested in the growth rate, probable failure
rate of my systems and how to obtain those measures as quickly,
quietly and accurately as possible.
That's the underlying question behind the first four topics in this
series.
I don't believe that even the best of traders is beyond trading
ruin, but if a portfolio is traded with, say 10 systems, each with a
likelihood of failure of say, 1/5, then the probability of portfolio
failure is known.
Isn't that the real risk that freelance traders should be
considering?
Of course it depends on what a trader considers failure.
I do have some reservations about MCS although it is early days for
me on the subject.
That is why I raised the subject in the forum; to get a second
opinion.
One of my queries is exactly eqivalent to your point; the basket of
trades in a back-tested sample is not precisely random.
One commercially available MCS software package puts all the trades
from a back-tested sample into a basket and draws from them randomly.
I have observed trade systems where the number of bull signals and
the win ratio of those signals is higher on bullish days, naturally
enough.
If that system is actually traded,capital limitations mean we can
only take, say one trade each day.
So the equity outcomes are biased slightly if all trades are
selected from a basket rather than selecting a single trade for each
day.
I therefore have a liking for rotational selection from amongst the
available period signals,exactly as provided by Ami.
The other concern I have with MCS is that if we have a basket of,
say three numbers, 1,2 and 3 the mean is 2.
If we *walk* through all possible combinations (equity curve) surely
the mean of the mean of the combinations will always and eventually
equal the mean of the sample, provided we run enough tests.
Of course, I am a naive mathematician so quite likely I am missing
something that the textbooks will sort out sometime in the future.
In the meantime my view is that the bias of statistics to solving
engineering, social engineering etc type problems has flowed over to
trading and that *we* may not be applying statistics to trading
problems in the best possible ways.
BrianB2.
--- In amibroker@xxxxxxxxxxxxxxx, "sebastiandanconia"
<sebastiandanconia@xxx> wrote:
>
> I only offer this as a consideration when using such testing, not
as a
> criticism of Monte Carlo Simulations. A subtle but significant
point
> (IMO) when using MCS: They may or may not be applicable to
> trading/investing, because the markets don't always behave
randomly.
>
> An example of when a MCS would clearly be appropriate: Let's say
you're
> a defense contractor manufacturing a part for the International
Space
> Station. The part is critical, but because of limitations in
> engineering technology it has a high failure rate, and there's no
way of
> forecasting in advance if a part will fail. However, although the
> failure rate is high, it's also very consistent. Until technology
> advances sufficiently the only practical solution is to keep
plenty of
> spares on-hand.
>
> A MCS could tell you what the optimal number of spares to keep on-
hand
> would be. The part failures are random, but MC could tell you the
> likelihood of two, three, five, ten, etc., consecutive failures.
You
> might determine that there would only be a 1/100,000 chance that 6
> spares in a row would fail, so you might advise NASA to stock at
least 6
> spares at all times.
>
> Some trading systems, though, will be successful because they take
> advantage of repeating sequences of events, not random events.
Business
> cycles go through a specific sequence, company growth follows a
certain
> pattern from infancy to maturity, price trends/reversals follow a
> sequence, etc. If trades based on reliable, repeating patterns are
> taken out of order by a MCS such that a massive drawdown or a
> bankrupting series of losers occurs, that can distort the value of
the
> trading method by putting the trades in an order that wouldn't
occur.
>
> Soapbox alert!:) Another reason that "Why does it work?" is such
an
> important question with trading systems, since a good answer to
that
> question can lead to a good answer for another important
question, "When
> WON'T it work?"
>
>
> S.
>
>
>
> --- In amibroker@xxxxxxxxxxxxxxx, "brian.z123" <brian.z123@> wrote:
> >
> > OT:margin of error example.
> >
> > As the trader is more interested in the general population of
future
> > trades than the test sample, what can be learnt from the sample?
> >
> > One answer is to trade the system for a decade or two and find
out.
> > Another option is to simulate decades or even centuries of
trading
> > by applying Monte Carlo analysis.
> > In laymans terms MCS is a computer generated, random walk through
> > *all*, of the possible trading outcomes based on the trading
sample
> > provided.
> > The result is a report or system profile that provides
statistics on
> > which to base our levels of trading confidence for the future.
> > There are other ways of sneaking a peak into a trading systems
> > future but MCS is the most commonly used.
> > I have developed my own system, that I don¡¦t want to headline
here
> > for various reasons, not the lest of which is that I can¡¦t
provide
> a
> > mathematical proof if called on to do so.
> >
> > Assuming that an MCS has been conducted on a sample of 50 trades
> > produced by a back-tested system and the report indicates that
the
> > meanW/meanL for the system over a large number of trading
> > simulations is 53/47. The StDev is 40% for both Wins and Losses.
> > How confident can we be in that result?
> >
> > From David Lanes statistical website:
> > http://davidmlane.com/hyperstat/A103397.html
> > The standard error of a statistic is the standard deviation of
the
> > sampling distribution of that statistic.
> > The formula for the standard error of the mean is:
> >
> > StErrorOfMeanPopulation = StDevPopulation/SqRt(N)Sample
> >
> > For any statistic:
> >
> > StErrorOfMeanPopulation(statistic) = StDevPopulation
(statistic)/SqRt
> > (N)Sample
> >
> > Applying the StdErrorMean equation to the example:
> >
> > Back-test sample size N = = 50,
> > MCS meanWin/meanLoss = = 53/47,
> > MCS Win StDev% = = 40%,
> > MCS Win StDev$ = = 40% x 53 = = 21.2,
> > MCS Loss StDev% = = 40%,
> > MCS Loss StDev$ = = 40% x 47 = = 18.8,
> >
> > StdError%Wins = = 40/SqRt(50) = = multiply mean by +/- 5.6 %,
> > Trading Win range = = 50 ¡V 56,
> > (min = = 53 x 0.943 = = 50, max = = 53 X 1.056 = = 56).
> >
> > The same result can be obtained using StDev as a number ($)
rather
> > than as a percentage.
> >
> > StdError$Wins = = 21.2/SqRt(50) = = +/- 3 = = Win range = = 47
+/-3
> > = = 50 -56 .
> >
> > Repeating the calculations for Losses shows the the mean Losses
> > range between 44 ¡V 50.
> >
> > I chose this extreme example to demonstrate the outcome for a
small
> > back-test sample with high volatility trades and a small win/loss
> > margin.
> >
> > If the same trading pattern were generated from a back-test
sample
> > of 2500 trades and the simulated meanWins and mean Losses each
had a
> > StDev of 10% the range for the margin of error would be:
> >
> > Wins 52.9 ¡V 53.1,
> > Losses 46.9 ¡V 47.1.
> >
> > This means that the we can be 95% confident the real mean values
are
> > somewhere within those ranges.
> > For a higher level of confidence the range will be greater.
> >
> > Resorting to the age-old teaching trick of asking the students
for
> > the answer while pretending to already know it yourself; can
anyone
> > in the forum tell me if this is the correct way to use StdError
when
> > applied to trading?
> >
> >
> >
> > BrianB2 ?º
> > --- In amibroker@xxxxxxxxxxxxxxx, "brian.z123" brian.z123@
> > wrote:
> > >
> > > Part1 of Project Based Training No1.
> > >
> > > The objective of the project is to introduce new traders to the
> > main
> > > concepts of system design/testing and demonstrate their
> > application
> > > in AmiBroker.
> > > At the same time it is hoped that the ideas presented will
provoke
> > > discussion and provide trading stimulation.
> > >
> > > All of the stages in the design process will not be
demonstrated
> > as
> > > most have already been covered elsewhere in the AmiBroker
support
> > > material.
> > >
> > > A basic understanding of the application of some statistical
> > methods
> > > to the trading environment is a pre-requisite.
> > > The opening topics address this need.
> > >
> > > To those who find the subject matter new *the project* will be
a
> > > workbook .
> > > To those who have experience in the subject it will be an
> > > opportunity to workshop.
> > >
> > > I would like to acknowledge my indebtedness to the academic
> > > community .
> > > I often refer to the material so generously interpreted for the
> > > layperson and made available at websites by academic
specialists,
> > > particularly those associated with Universities.
> > >
> > >
*******************************************************************
> > > Margin of Error.
> > >
> > > Back-testing of historical data provides traders with a
sample,
> > > typical of the trade they are testing. From that sample they
make
> > > inferences about the larger group, or population, of all past
> > trades
> > > and future trades, of the same type, that were not included in
> > their
> > > test window.
> > > Despite the fact that the people who teach them to back-test
also
> > > teach them that the past can not predict the future, some
continue
> > > to act as if it can.
> > >
> > > If the past can't predict the future. How can anyone trade with
> > > confidence?
> > >
> > > The answer is that while the future can't be predicted, the
> > > likelihood of some mathematically defined outcomes can be
> > predicted
> > > with a degree of confidence.
> > > Statistics is the mathematical discipline that manages that
very
> > > well.
> > >
> > > The caveat is that to apply statistical methods to trading
> > samples,
> > > the assumption is made that they are the result of a random
> > process.
> > > Where the trading system chosen is biased to non-random
behaviour
> > it
> > > will be prone to failure if the market acts contrary to that
bias.
> > >
> > > For that reason system traders are faced with a choice between
> > > attempting to define market behaviour e.g. a trend, and pick a
> > > system to suit that, or search for a universal signal that is
> > > consistent irrespective of any assumed market bias.
> > >
> > > If statistics can predict the likelihood of future trading
> > outcomes,
> > > how accurate will it be?
> > >
> > > *Standard error* or *margin of error* offers traders a
solution
> > but
> > > they are not subjects that are often discussed.
> > >
> > > In his book ,*Design, Testing, and Optimisation of Trading
> > Systems*
> > > (John Wiley & Sons, 1992), Robert Pardo raises the issue of the
> > > accuracy of trading *predictions* based on the size of the
sample
> > > used:
> > >
> > > * The sample size must be large enough to allow the trading
system
> > > to generate a statistically significant sample of trades.
> > > A sample of one trade is certainly insignificant, whereas a
sample
> > > of 50 trades or more is generally adequate.*
> > >
> > > He uses Standard Error as a measure of significance:
> > >
> > > StdError = = 1/SquareRoot(sample size),
> > >
> > > 1/SqRt(50) = = 14.1%.
> > >
> > > There is little by way of further explanation provided.
> > >
> > > Applying the formula to a greater number of samples:
> > >
> > > Where N = = the number of trades in the sample
> > >
> > > StdError factor = = 1/SqRt(N)
> > > StdError% = 1/SqRt(N) * 100
> > >
> > > If N = = 2500 the StdError% = = 1/SqRt(2500) * 100 = = +/- 2%
> > > If N = = 10000 the StdError% = = 1/SqRt(10000) * 100 = = +/-
1%
> > >
> > > A trade sample of 10000 to provide statistical accuracy of 1%
is
> > not
> > > easily achievable for traders, although a lot easier than
> > accurately
> > > surveying the eye colour of Polar Bears.
> > >
> > > Pardos equation is in fact, a rounding of the StdError equation
> > for
> > > a 95% level of confidence:
> > >
> > > Margin of error at 99% confidence = = 1.29/SqRt(N)
> > > Margin of error at 95% confidence = = 0.98/SqRt(N)
> > > Margin of error at 90% confidence = = 0.82/SqRt(N)
> > >
> > > Later in the project I will use a basic random number
generator,
> > > within Xcel, to provide a visual aid that traders can use to
> > > understand the *sample* concept and decide for themselves what
> > > constitutes an adequate sample.
> > >
> > > Wikipedia provides some additional clarity on the subject:
> > >
> > > http://en.wikipedia.org/wiki/Margin_of_error
> > >
> > > *The margin of error expresses the amount of the random
variation
> > > underlying a survey's results. This can be thought of as a
measure
> > > of the variation one would see in reported percentages if the
same
> > > poll were taken multiple times. The larger the margin of error,
> > the
> > > less confidence one has that the poll's reported percentages
are
> > > close to the "true" percentages, that is the percentages in the
> > > whole population.*
> > >
> > > *An interesting mathematical fact is that the margin of error
> > > depends only on the sample size and not on the population size,
> > > provided that the population is significantly larger than the
> > sample
> > > size, and provided a simple random sample is used. Thus for
> > > instance¡K¡K.the running example with 1,013 random
> samples¡K¡Kwould
> > > yield essentially the same margin of error (4% with a 99%
level of
> > > confidence) regardless of whether the
> population¡K¡K¡K.consisted of
> > > 100,000 or 100,000,000.*
> > >
> > > In short the tail of the trading system sample is swinging the
> > > trading system cat.
> > >
> > > BrianB2
> > >
> > > The material contained in this topic is for educational and
> > > discussion use only.
> > > It is not intended as financial advice and should not be
construed
> > > as such.
> > > The author is not an accredited academic or financial advisor.
> > >
> >
>
Content-Description: "AVG certification"
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.409 / Virus Database: 268.14.11/542 - Release Date: 11/20/2006
|