|
PureBytes Links
Trading Reference Links
|
Fred,
Mr Reality is fine by me.
Like any trader, I want the best I can get and as soon as I get it I
want more.
I picked a number, 1/5, to demonstrate the principle of dispersing
risk in a trading portfolio as opposed to an investment portfolio
based on standard portfolio theory.
Of course a trading portfolio could be only one part of a broader
investment portfolio.
1/5 with the trading portfolio spread over 10 systems has 1 chance
in 9,765,625 of all of the systems coming to ruin at the same time.
The probability of ruin chosen for any system depends on the risk
tolerance of the individual.
1/5 might be the chance of a 20% drawdown for the system.
Do you disagree with that or think that theory is incorrect?
Thanks for the summary of IO which fits well with the overall
project and puts in into its context.
That is one of the reasons I kept away from backtesting or
optimising the calender effect in the *project* as I believe IO and
other places in Ami are the best place to get that info.
I noted your commitment to OOS and WF as a *look into the future
method* and gave it a decent weighting.
Some trading commentators claim that walk forward isn't actually
necessary, but it would be a brave trader who takes that path,
especially after optimising.
Privately a part of me is criticising the walk forward philosophy.
There is no logical reason why the results of an IS test are any
more or less valid than the results of an OOS test.
If one OOS test returned + 50% drawdown or - 50% drawdown it doesn't
say that much about the system.
It only says something about that one single OOS test of however
many samples.
I don't think I am brave enough to attempt it though.
I might layout the arguments in this topic but more than likely I
will hold fire until after I have made a thorough study of IO.
BrianB2.
--- In amibroker@xxxxxxxxxxxxxxx, "Fred" <ftonetti@xxx> wrote:
>
> I am hardly Mr. Mathematician ... I do however try to be Mr.
> Reality ...
>
> I'll ask a question relative to your previous post and try to
answer
> your question regarding the post previous to that ...
>
> The question is ... Would you really trade or incorporate into a
> group of systems to trade one that you thought via some
methodology
> had a 1 in 5 chance of failure ?
>
> The answer to your question about what tells us something about
what
> our system is likely to do out of sample is simply enough to test
it
> out of sample ... This is ( for good personal reasons ) why I
built
> automated OOS and WF testing into IO ... WF testing is a royal
PITA
> to do manually as the process essentially involves ...
>
> Some form of optimization ...
> Rolling the end date forward by some amount of time or by some
number
> of OOS trades ...
> Running a Backtest
> Viewing and/or better yet saving the OOS backtest results ...
> Possibly moving the begin date forward ( for rolling as opposed to
> anchored WF )
> Go back to step 1.
>
> If there is an automated process that allows you to save the
results
> from the OOS backtest ( as IO does ) then after a succession of WF
> tests you have a complete equity curve consisting of the original
IS
> and all OOS segments that show as a whole end to end much of which
is
> OOS ...
>
> --- In amibroker@xxxxxxxxxxxxxxx, "brian.z123" <brian.z123@> wrote:
> >
> > Hello Chuck,
> >
> > We don't have to be an expert for our opinions to be worthwhile.
> > The point of the forum is to bring the weight of our collective
> > expertise to bear on a problem.
> > Everyone brings something to the party.
> > Thinking is one activity where the whole is greater than the sum
of
> > the parts.
> > Fortunately we have people like Fred to keep us on the
mathematical
> > and logico straight and narrow.
> >
> > What I am talking about is not the merit of the various trade
> > evaluators but the methods we can use to obtain the numbers they
> are
> > based on and what confidence we can have in them and the
> > computations they produce.
> >
> > I agree that it is very important to know the volatility of our
> > systems.
> > How to measure that is another point altogether.
> > I am not a big fan of Sharpe ratios but that would be a good
topic
> > all by itself.
> >
> > BrianB2.
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "cstrader" <cstrader232@>
> > wrote:
> > >
> > > Hi Brian:
> > >
> > > I'm not an expert, but it seems that your analysis is
correct.
> > The Sharpe
> > > Ratio basically does the same thing -- how variable are the
> trades
> > compared
> > > to their standard deviation within a sample, or to the
standard
> > error of
> > > their samples? Of course this all assumes that the
effectiveness
> > of the
> > > system does not change over time...
> > >
> > > chuck
> > >
> > >
> > >
> > > ----- Original Message -----
> > > From: "brian.z123" <brian.z123@>
> > > To: <amibroker@xxxxxxxxxxxxxxx>
> > > Sent: Tuesday, November 07, 2006 2:21 AM
> > > Subject: [amibroker] Re: Margin of Error
> > >
> > >
> > > OT:margin of error example.
> > >
> > > As the trader is more interested in the general population of
> > future
> > > trades than the test sample, what can be learnt from the
sample?
> > >
> > > One answer is to trade the system for a decade or two and find
> out.
> > > Another option is to simulate decades or even centuries of
> trading
> > > by applying Monte Carlo analysis.
> > > In laymans terms MCS is a computer generated, random walk
through
> > > *all*, of the possible trading outcomes based on the trading
> sample
> > > provided.
> > > The result is a report or system profile that provides
statistics
> > on
> > > which to base our levels of trading confidence for the future.
> > > There are other ways of sneaking a peak into a trading systems
> > > future but MCS is the most commonly used.
> > > I have developed my own system, that I don¡¦t want to headline
> here
> > > for various reasons, not the lest of which is that I can¡¦t
> > provide a
> > > mathematical proof if called on to do so.
> > >
> > > Assuming that an MCS has been conducted on a sample of 50
trades
> > > produced by a back-tested system and the report indicates that
the
> > > meanW/meanL for the system over a large number of trading
> > > simulations is 53/47. The StDev is 40% for both Wins and
Losses.
> > > How confident can we be in that result?
> > >
> > > From David Lanes statistical website:
> > > http://davidmlane.com/hyperstat/A103397.html
> > > The standard error of a statistic is the standard deviation of
the
> > > sampling distribution of that statistic.
> > > The formula for the standard error of the mean is:
> > >
> > > StErrorOfMeanPopulation = StDevPopulation/SqRt(N)Sample
> > >
> > > For any statistic:
> > >
> > > StErrorOfMeanPopulation(statistic) = StDevPopulation
> > (statistic)/SqRt
> > > (N)Sample
> > >
> > > Applying the StdErrorMean equation to the example:
> > >
> > > Back-test sample size N = = 50,
> > > MCS meanWin/meanLoss = = 53/47,
> > > MCS Win StDev% = = 40%,
> > > MCS Win StDev$ = = 40% x 53 = = 21.2,
> > > MCS Loss StDev% = = 40%,
> > > MCS Loss StDev$ = = 40% x 47 = = 18.8,
> > >
> > > StdError%Wins = = 40/SqRt(50) = = multiply mean by +/- 5.6 %,
> > > Trading Win range = = 50 ¡V 56,
> > > (min = = 53 x 0.943 = = 50, max = = 53 X 1.056 = = 56).
> > >
> > > The same result can be obtained using StDev as a number ($)
rather
> > > than as a percentage.
> > >
> > > StdError$Wins = = 21.2/SqRt(50) = = +/- 3 = = Win range = =
47
> +/-
> > 3
> > > = = 50 -56 .
> > >
> > > Repeating the calculations for Losses shows the the mean Losses
> > > range between 44 ¡V 50.
> > >
> > > I chose this extreme example to demonstrate the outcome for a
> small
> > > back-test sample with high volatility trades and a small
win/loss
> > > margin.
> > >
> > > If the same trading pattern were generated from a back-test
sample
> > > of 2500 trades and the simulated meanWins and mean Losses each
> had
> > a
> > > StDev of 10% the range for the margin of error would be:
> > >
> > > Wins 52.9 ¡V 53.1,
> > > Losses 46.9 ¡V 47.1.
> > >
> > > This means that the we can be 95% confident the real mean
values
> > are
> > > somewhere within those ranges.
> > > For a higher level of confidence the range will be greater.
> > >
> > > Resorting to the age-old teaching trick of asking the students
for
> > > the answer while pretending to already know it yourself; can
> anyone
> > > in the forum tell me if this is the correct way to use
StdError
> > when
> > > applied to trading?
> > >
> > >
> > >
> > > BrianB2 fº
> > > --- In amibroker@xxxxxxxxxxxxxxx, "brian.z123" <brian.z123@>
> > > wrote:
> > > >
> > > > Part1 of Project Based Training No1.
> > > >
> > > > The objective of the project is to introduce new traders to
the
> > > main
> > > > concepts of system design/testing and demonstrate their
> > > application
> > > > in AmiBroker.
> > > > At the same time it is hoped that the ideas presented will
> > provoke
> > > > discussion and provide trading stimulation.
> > > >
> > > > All of the stages in the design process will not be
demonstrated
> > > as
> > > > most have already been covered elsewhere in the AmiBroker
> support
> > > > material.
> > > >
> > > > A basic understanding of the application of some statistical
> > > methods
> > > > to the trading environment is a pre-requisite.
> > > > The opening topics address this need.
> > > >
> > > > To those who find the subject matter new *the project* will
be a
> > > > workbook .
> > > > To those who have experience in the subject it will be an
> > > > opportunity to workshop.
> > > >
> > > > I would like to acknowledge my indebtedness to the academic
> > > > community .
> > > > I often refer to the material so generously interpreted for
the
> > > > layperson and made available at websites by academic
> specialists,
> > > > particularly those associated with Universities.
> > > >
> > > >
> >
*******************************************************************
> > > > Margin of Error.
> > > >
> > > > Back-testing of historical data provides traders with a
sample,
> > > > typical of the trade they are testing. From that sample they
> make
> > > > inferences about the larger group, or population, of all past
> > > trades
> > > > and future trades, of the same type, that were not included
in
> > > their
> > > > test window.
> > > > Despite the fact that the people who teach them to back-test
> also
> > > > teach them that the past can not predict the future, some
> > continue
> > > > to act as if it can.
> > > >
> > > > If the past can't predict the future. How can anyone trade
with
> > > > confidence?
> > > >
> > > > The answer is that while the future can't be predicted, the
> > > > likelihood of some mathematically defined outcomes can be
> > > predicted
> > > > with a degree of confidence.
> > > > Statistics is the mathematical discipline that manages that
very
> > > > well.
> > > >
> > > > The caveat is that to apply statistical methods to trading
> > > samples,
> > > > the assumption is made that they are the result of a random
> > > process.
> > > > Where the trading system chosen is biased to non-random
> behaviour
> > > it
> > > > will be prone to failure if the market acts contrary to that
> > bias.
> > > >
> > > > For that reason system traders are faced with a choice
between
> > > > attempting to define market behaviour e.g. a trend, and pick
a
> > > > system to suit that, or search for a universal signal that is
> > > > consistent irrespective of any assumed market bias.
> > > >
> > > > If statistics can predict the likelihood of future trading
> > > outcomes,
> > > > how accurate will it be?
> > > >
> > > > *Standard error* or *margin of error* offers traders a
solution
> > > but
> > > > they are not subjects that are often discussed.
> > > >
> > > > In his book ,*Design, Testing, and Optimisation of Trading
> > > Systems*
> > > > (John Wiley & Sons, 1992), Robert Pardo raises the issue of
the
> > > > accuracy of trading *predictions* based on the size of the
> sample
> > > > used:
> > > >
> > > > * The sample size must be large enough to allow the trading
> > system
> > > > to generate a statistically significant sample of trades.
> > > > A sample of one trade is certainly insignificant, whereas a
> > sample
> > > > of 50 trades or more is generally adequate.*
> > > >
> > > > He uses Standard Error as a measure of significance:
> > > >
> > > > StdError = = 1/SquareRoot(sample size),
> > > >
> > > > 1/SqRt(50) = = 14.1%.
> > > >
> > > > There is little by way of further explanation provided.
> > > >
> > > > Applying the formula to a greater number of samples:
> > > >
> > > > Where N = = the number of trades in the sample
> > > >
> > > > StdError factor = = 1/SqRt(N)
> > > > StdError% = 1/SqRt(N) * 100
> > > >
> > > > If N = = 2500 the StdError% = = 1/SqRt(2500) * 100 = = +/- 2%
> > > > If N = = 10000 the StdError% = = 1/SqRt(10000) * 100 = = +/-
1%
> > > >
> > > > A trade sample of 10000 to provide statistical accuracy of
1% is
> > > not
> > > > easily achievable for traders, although a lot easier than
> > > accurately
> > > > surveying the eye colour of Polar Bears.
> > > >
> > > > Pardos equation is in fact, a rounding of the StdError
equation
> > > for
> > > > a 95% level of confidence:
> > > >
> > > > Margin of error at 99% confidence = = 1.29/SqRt(N)
> > > > Margin of error at 95% confidence = = 0.98/SqRt(N)
> > > > Margin of error at 90% confidence = = 0.82/SqRt(N)
> > > >
> > > > Later in the project I will use a basic random number
generator,
> > > > within Xcel, to provide a visual aid that traders can use to
> > > > understand the *sample* concept and decide for themselves
what
> > > > constitutes an adequate sample.
> > > >
> > > > Wikipedia provides some additional clarity on the subject:
> > > >
> > > > http://en.wikipedia.org/wiki/Margin_of_error
> > > >
> > > > *The margin of error expresses the amount of the random
> variation
> > > > underlying a survey's results. This can be thought of as a
> > measure
> > > > of the variation one would see in reported percentages if
the
> > same
> > > > poll were taken multiple times. The larger the margin of
error,
> > > the
> > > > less confidence one has that the poll's reported percentages
are
> > > > close to the "true" percentages, that is the percentages in
the
> > > > whole population.*
> > > >
> > > > *An interesting mathematical fact is that the margin of error
> > > > depends only on the sample size and not on the population
size,
> > > > provided that the population is significantly larger than the
> > > sample
> > > > size, and provided a simple random sample is used. Thus for
> > > > instance¡K¡K.the running example with 1,013 random
> > samples¡K¡Kwould
> > > > yield essentially the same margin of error (4% with a 99%
level
> > of
> > > > confidence) regardless of whether the
> population¡K¡K¡K.consisted
> > of
> > > > 100,000 or 100,000,000.*
> > > >
> > > > In short the tail of the trading system sample is swinging
the
> > > > trading system cat.
> > > >
> > > > BrianB2
> > > >
> > > > The material contained in this topic is for educational and
> > > > discussion use only.
> > > > It is not intended as financial advice and should not be
> > construed
> > > > as such.
> > > > The author is not an accredited academic or financial
advisor.
> > > >
> > >
> > >
> > >
> > >
> > >
> > > Please note that this group is for discussion between users
only.
> > >
> > > To get support from AmiBroker please send an e-mail directly to
> > > SUPPORT {at} amibroker.com
> > >
> > > For NEW RELEASE ANNOUNCEMENTS and other news always check
DEVLOG:
> > > http://www.amibroker.com/devlog/
> > >
> > > For other support material please check also:
> > > http://www.amibroker.com/support.html
> > >
> > > Yahoo! Groups Links
> > >
> >
>
Content-Description: "AVG certification"
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.409 / Virus Database: 268.14.11/542 - Release Date: 11/20/2006
|