|
PureBytes Links
Trading Reference Links
|
There is as stated also an AmiBroker-TS ( Trading Systems )
forum ... as well as several others i.e. Beta ... DLL ... AT ...
--- In amibroker@xxxxxxxxxxxxxxx, "sebastiandanconia"
<sebastiandanconia@xxx> wrote:
>
> "... I could post a variety of examples to refute your
assumptions ...
> But
> as stated by someone else earlier ... THIS ISN'T THE FORUM FOR
IT ...
> Take it up in AB-TS ... This forum is supposed to be about AB ..."
>
> Fred, I'm regretting my "aiding and abetting" of this topic here,
and
> related ones, too. This forum is where I come to for help with AB
> coding and software solutions and I think the overwhelming
majority of
> other members do, too. It just isn't the place for extended
> discussions like this one, and as for myself I'm going to make a
serious
> effort to stay more on-topic.
>
> As an alternative for posters who have an interest in topics
related to
> mechanical systems and valid testing concepts, I've found this
site to
> be absolutely first-rate.
>
> http://www.tradingblox.com/forum/
>
>
> Luck,
>
> Sebastian
>
>
> --- In amibroker@xxxxxxxxxxxxxxx, "Fred" <ftonetti@> wrote:
> >
> > Brian,
> >
> > Your assumptions have moved for the outrageous ( your word ) to
the
> > absurd ( my word ) ... They are based only on a few loose things
> > rolling around in your head that you think should be true. If
you
> > want to go the route you're on, fine, do it ... but to advise
others
> > to do the same is imho at best DANGEROUS ...
> >
> > I could post a variety of examples to refute your
assumptions ... But
> > as stated by someone else earlier ... THIS ISN'T THE FORUM FOR
IT ...
> > Take it up in AB-TS ... This forum is supposed to be about AB ...
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "brian.z123" brian.z123@ wrote:
> > >
> > > Fred,
> > >
> > >
> > > I put forward the following proposition, not as a method, but
> > rather
> > > as an interesting discussion point.
> > >
> > > My proposition is that OOS testing after optimisation is not
> > actually
> > > OOS testing at all.
> > >
> > > Break some historical data into three equal segments and
conduct
> > > intitial testing/optimisation on the first segment.
> > > This is not testing as such and the data used can be
considered as
> > > optimisation or design data.
> > > After the optimisation the top model can be tested using
segment
> > two.
> > > I claim that this is not OOS but actually the first test of
the top
> > > model.
> > > No harm is done if the top model is disappointing in tests and
the
> > > second top model is tested on the second segment of data or
even if
> > we
> > > go back and re-optimise on the first segment of data to obtain
some
> > > new top models.
> > > The data doesn't know we have optimised on it.
> > > It suddenly doesn't become data-non-gratus just because our
> > computer
> > > software has tip-toed over it once, twice or even a thousand
times.
> > >
> > > Once an optimised model has been correctly tested in out of
> > > optimisation data, it can be statistically evaluated and then
> > traded
> > > with confidence, providing it is part of a balanced freelance
> > trading
> > > portfolio.
> > >
> > > If the system is tested on the third set of data, that would
> > > constitute an OOS test, but no one ever does that do they?
> > >
> > > Further to that, the equity curve obtained from the OOS test
will
> > > *always* be within the range predicted by the test profile of a
> > > correctly analysed system, and yes, sometimes it might not
look so
> > > pretty.
> > > The exception there is the occasional equity curve outlier, but
> > hey,
> > > nothings perfect.
> > >
> > > My second outlandish proposition is that the dangers of over-
> > fitting
> > > during optimisation are over-emphasised.
> > > If the outcome of the trade system test is a dataset with an
> > adequate
> > > number of samples then it will be a true test and definitely
will
> > not
> > > be a result of over-fitting.
> > > The corrollary of that is that if a system has that many rules
that
> > > after back-testing 10 X 250 daily bars for 2000 symbols
(5million
> > > datapoints) it only produces 5 signals it is obvious that
something
> > is
> > > wrong.
> > >
> > > Genuinely significant events that occur rarely require massive
> > amounts
> > > of data to produce a resonable number of signals, so the data
> > becomes
> > > suspect anyway and as well as that it is no fun to trade a
system
> > when
> > > you have to wait for a leap year to get in.
> > >
> > > Most of the time over-fitting simply receives the blame for
> > incorrect
> > > testing and evaluation.
> > >
> > > BrianB2.
> > >
> > >
> > >
> > >
> > > --- In amibroker@xxxxxxxxxxxxxxx, "Fred" <ftonetti@> wrote:
> > > >
> > > > OOS and/or WF Testing is not a concept invented in or for IO
so
> > that
> > > > particular piece of software is not really the issue per se
> > except
> > > that
> > > > IO has facilitated making it considerably easier to perform.
> > > >
> > > > "If one OOS test had a 50% drawdown it doesn't say that much
> > about
> > > the
> > > > system. It only says something about that one single OOS
test of
> > > > however many samples."
> > > >
> > > > It doesn't ? ... It speaks volumes to me ... From my
perspective,
> > > > decent in sample performance, whether or not one applies MCS
> > after
> > > the
> > > > fact, is not the end of system testing, it is only a
milestone
> > along
> > > > the way to developing a system that MIGHT be tradable ...
> > > >
> > >
> >
>
Content-Description: "AVG certification"
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.409 / Virus Database: 268.14.11/542 - Release Date: 11/20/2006
|