|
PureBytes Links
Trading Reference Links
|
Fred,
That' alright.
I don't mind your comments at all.
I'm enjoying the joust.
I have more ammunition up my sleeve yet.
I am pleased that you are reiterating my warning to others not to
try this at home.
I said it was a discussion and not a monologue or a trading method.
We have to discuss these things somewhere.
I am not advising anyone to do anything.
If they can't think for themselves they should give up trading now.
I provided a formal warning at the foot of the initial post that I
was not providing financial or trading advice.
I haven't placed any restrictions on myself that prevent me saying
absurd things.
No, I won't be changing forums.
I apologise that I forgot to put OT before the topic, but people are
smart enough to figure that out for themselves and anyone who
doesn't want to follow the topic would have bypassed it a long way
back.
I don't apologise for discussing it here.
It is far more appropriate to AmiBokerists, who are also traders to
the very last man or woman, than a lot of the other stuff that is
posted in this forum.
BrianB2.
--- In amibroker@xxxxxxxxxxxxxxx, "Fred" <ftonetti@xxx> wrote:
>
> Brian,
>
> Your assumptions have moved for the outrageous ( your word ) to
the
> absurd ( my word ) ... They are based only on a few loose things
> rolling around in your head that you think should be true. If you
> want to go the route you're on, fine, do it ... but to advise
others
> to do the same is imho at best DANGEROUS ...
>
> I could post a variety of examples to refute your assumptions ...
But
> as stated by someone else earlier ... THIS ISN'T THE FORUM FOR
IT ...
> Take it up in AB-TS ... This forum is supposed to be about AB ...
>
> --- In amibroker@xxxxxxxxxxxxxxx, "brian.z123" <brian.z123@> wrote:
> >
> > Fred,
> >
> >
> > I put forward the following proposition, not as a method, but
> rather
> > as an interesting discussion point.
> >
> > My proposition is that OOS testing after optimisation is not
> actually
> > OOS testing at all.
> >
> > Break some historical data into three equal segments and conduct
> > intitial testing/optimisation on the first segment.
> > This is not testing as such and the data used can be considered
as
> > optimisation or design data.
> > After the optimisation the top model can be tested using segment
> two.
> > I claim that this is not OOS but actually the first test of the
top
> > model.
> > No harm is done if the top model is disappointing in tests and
the
> > second top model is tested on the second segment of data or even
if
> we
> > go back and re-optimise on the first segment of data to obtain
some
> > new top models.
> > The data doesn't know we have optimised on it.
> > It suddenly doesn't become data-non-gratus just because our
> computer
> > software has tip-toed over it once, twice or even a thousand
times.
> >
> > Once an optimised model has been correctly tested in out of
> > optimisation data, it can be statistically evaluated and then
> traded
> > with confidence, providing it is part of a balanced freelance
> trading
> > portfolio.
> >
> > If the system is tested on the third set of data, that would
> > constitute an OOS test, but no one ever does that do they?
> >
> > Further to that, the equity curve obtained from the OOS test
will
> > *always* be within the range predicted by the test profile of a
> > correctly analysed system, and yes, sometimes it might not look
so
> > pretty.
> > The exception there is the occasional equity curve outlier, but
> hey,
> > nothings perfect.
> >
> > My second outlandish proposition is that the dangers of over-
> fitting
> > during optimisation are over-emphasised.
> > If the outcome of the trade system test is a dataset with an
> adequate
> > number of samples then it will be a true test and definitely
will
> not
> > be a result of over-fitting.
> > The corrollary of that is that if a system has that many rules
that
> > after back-testing 10 X 250 daily bars for 2000 symbols
(5million
> > datapoints) it only produces 5 signals it is obvious that
something
> is
> > wrong.
> >
> > Genuinely significant events that occur rarely require massive
> amounts
> > of data to produce a resonable number of signals, so the data
> becomes
> > suspect anyway and as well as that it is no fun to trade a
system
> when
> > you have to wait for a leap year to get in.
> >
> > Most of the time over-fitting simply receives the blame for
> incorrect
> > testing and evaluation.
> >
> > BrianB2.
> >
> >
> >
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "Fred" <ftonetti@> wrote:
> > >
> > > OOS and/or WF Testing is not a concept invented in or for IO
so
> that
> > > particular piece of software is not really the issue per se
> except
> > that
> > > IO has facilitated making it considerably easier to perform.
> > >
> > > "If one OOS test had a 50% drawdown it doesn't say that much
> about
> > the
> > > system. It only says something about that one single OOS test
of
> > > however many samples."
> > >
> > > It doesn't ? ... It speaks volumes to me ... From my
perspective,
> > > decent in sample performance, whether or not one applies MCS
> after
> > the
> > > fact, is not the end of system testing, it is only a milestone
> along
> > > the way to developing a system that MIGHT be tradable ...
> > >
> >
>
Content-Description: "AVG certification"
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.409 / Virus Database: 268.14.11/542 - Release Date: 11/20/2006
|