|
PureBytes Links
Trading Reference Links
|
It's not about margin of error ... It's about the confidence level or
lack thereof for what the system will do with data it has not yet
seen.
For a simple OOS test I would probably split the data in half ...
optimize the system based on the first half of the data and then test
the second half to observe what the OOS performance metrics were ...
For a more thorough test, I would, as I outlined earlier, take the
minimum amount of data I could to generate what I felt were a
sufficient number of trades and then I would ...
- Optimize the system on that data
- Test some period following the IS as OOS
- Roll the end date forward to the end of the OOS
- Possibly roll the begin date forward by the same amount
- Go back to the Optimize step
I would continue to do this until I ran out of data ...
This would show me the anchored or rolling WF / OOS results for the
greatest period of time and assuming the system was worth trading
leave me with parameter values that could be used going forward.
While I take his book as more of a guide then a bible, you might want
to read Pardo's relatively short ... Design, Testing & Optimization
of Trading Systems ...
--- In amibroker@xxxxxxxxxxxxxxx, "brian.z123" <brian.z123@xxx> wrote:
>
> The question is:
>
> If I have 10 years of data and that provides 500 Monday samples for
> a Calender Effect study would the margin for error be less or more
> if I tested the sample in one segment (500 trades) or undertook 5
> tests with 100 samples and then combined the results in some way?
>
> If so, what is the best way to use the 5 samples to get the
> population outcomes.
>
> No offence taken if you are too busy to answer.
>
> BrianB2.
>
> --- In amibroker@xxxxxxxxxxxxxxx, "quanttrader714"
> <quanttrader714@> wrote:
> >
> > One way to approach this is with a form of the block bootstrap, by
> > resampling blocks of consecutive observations of random length
> with
> > replacement.
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "sebastiandanconia"
> > <sebastiandanconia@> wrote:
> > >
> > > I only offer this as a consideration when using such testing,
> not as a
> > > criticism of Monte Carlo Simulations. A subtle but significant
> point
> > > (IMO) when using MCS: They may or may not be applicable to
> > > trading/investing, because the markets don't always behave
> randomly.
> > >
> > > An example of when a MCS would clearly be appropriate: Let's
> say you're
> > > a defense contractor manufacturing a part for the International
> Space
> > > Station. The part is critical, but because of limitations in
> > > engineering technology it has a high failure rate, and there's
> no way of
> > > forecasting in advance if a part will fail. However, although
> the
> > > failure rate is high, it's also very consistent. Until
> technology
> > > advances sufficiently the only practical solution is to keep
> plenty of
> > > spares on-hand.
> > >
> > > A MCS could tell you what the optimal number of spares to keep
> on-hand
> > > would be. The part failures are random, but MC could tell you
> the
> > > likelihood of two, three, five, ten, etc., consecutive
> failures. You
> > > might determine that there would only be a 1/100,000 chance
that
> 6
> > > spares in a row would fail, so you might advise NASA to stock
at
> least 6
> > > spares at all times.
> > >
> > > Some trading systems, though, will be successful because they
> take
> > > advantage of repeating sequences of events, not random events.
> Business
> > > cycles go through a specific sequence, company growth follows a
> certain
> > > pattern from infancy to maturity, price trends/reversals follow
a
> > > sequence, etc. If trades based on reliable, repeating patterns
> are
> > > taken out of order by a MCS such that a massive drawdown or a
> > > bankrupting series of losers occurs, that can distort the value
> of the
> > > trading method by putting the trades in an order that wouldn't
> occur.
> > >
> > > Soapbox alert!:) Another reason that "Why does it work?" is
> such an
> > > important question with trading systems, since a good answer to
> that
> > > question can lead to a good answer for another important
> question, "When
> > > WON'T it work?"
> > >
> > >
> > > S.
> >
>
Content-Description: "AVG certification"
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.409 / Virus Database: 268.14.11/542 - Release Date: 11/20/2006
|