--- In
amibroker@xxxxxxxxxxxxxxx, ftonetti@xxx wrote:
>
> Near Random ?
>
> You're kidding right ? ... I'd thought you had beeen around longer then that ...
>
> On another level the number of trades has no relevance in calculating K-Ratio and as calculated in AB is useless for anything but systems that trade constant dollar amounts ... By definition that would imply no degree of success over a long period of time.
>
> ----- Original Message -----
> From: brian_z11
> Date: Sunday, May 10, 2009 8:58 pm
> Subject: [amibroker] Re: Expectancy - and related--specifically K-rato
> To:
amibroker@xxxxxxxxxxxxxxx
>
> > My understanding of Bob Pardo's book is that he feels
> > that the length of the
> > out-of-sample period can be determined by a calculation based on
> > the length
> > of the in-sample period. If that were true, then the ratio
> > would be a
> > little easier to compute. But, in my experience, there is no
> > relationshipbetween the length of the out-of-sample period and
> > the length of the
> > in-sample period. And gathering the data and performing the
> > calculation of
> > the ratio for some objective functions would be difficult.
> >
> > I don't believe that it would be worth the effort but setting
> > tbe BT the task of collecting the same number of trades in the
> > OOS test as were collected in the IS test is the way to
> > standardise and simplify calculating this ratio.
> >
> > N == the number of closed trades in a test run;
> > Where IS(N) == OOS(N) the ratio IS(N bars)/OOS(N bars) will
> > always be defined as a probability distribution.
> >
> > No constant relationsip, between IS and OOS length (in bars) can
> > exist because of the (near) random nature of the markets.
> >
> > Believe that such a relationship does exist, and hence can be
> > calculated/exploited, is an artefact of the belief that the
> > markets are non random i.e. have a structure that we can
> > identify, model and therefore use to make predictions about the
> > future of the markets.
> >
> > Any attempt to improve OOS performance by 'tuning' the length of
> > the sample periods is a futile excercise.
> >
> > The exception to this, as you point out in your book, is that
> > the markets do change (they are not quite random .... human
> > behaviour leaves a faint trail of non randomness) and so current
> > data may be more relevant than non- current data e.g. increased
> > use of computers, and access to information, by traders may have
> > changed the markets in the last decade or two.
> >
> > Whether these changes are strong enough, and occur over short
> > enough time periods or identifiable time periods, to justify the
> > view that we can/must account for this, in our design processes,
> > is arguable i.e. it is questionable that we can "synchronise the
> > market and our trading system by shortening (varying) the time
> > periods used in our walk forward testing", as you suggested in
> > your QTS book (P261).
> >
> > Granted that we are attempting to synchronise our systems to
> > market behaviour but this is done within our system rules, not
> > by altering the N bars tested.
> >
> > The fundamental premise of trading is that the system will
> > perform, in the future, irrespective of market conditions or the
> > time period involved.
> >
> > The OOS walk through is a test to find out if the patterns we
> > identified in the IS data still exist in the future and that the
> > code we used to synchronise to that pattern(s) is effective.
> >
> > The only time that will be time dependent is when the patterns
> > are time based e.g. the number of times the price of oil goes up
> > on the first day of the month is statistically significant.
> >
> > Very few exploitable inefficiencies in the markets are time based.
> >
> >
> > On another point:
> >
> > Theoretically the OOS results should be better/worse than the IS
> > tests on a 50/50 basis.
> >
> > Continual skewing of that ratio to the downside is an indicator
> > of curve fitting at the IS stage?
> >
> >
> >
> >
> >
> > --- In
amibroker@xxxxxxxxxxxxxxx, Howard B wrote:
> > >
> > > Greetings all --
> > >
> > > In-sample results and out-of-sample results can be, and
> > usually are, very
> > > different in their characteristics.
> > >
> > > My experience is that the ratio (OOS/(IS+OOS)), where these
> > are the
> > > in-sample and out-of-sample results, is difficult to compute
> > and often even
> > > difficult to define. I have not found it to be of value in
> > estimating the
> > > future performance of the system.
> > >
> > > My understanding of Bob Pardo's book is that he feels that the
> > length of the
> > > out-of-sample period can be determined by a calculation based
> > on the length
> > > of the in-sample period. If that were true, then the ratio
> > would be a
> > > little easier to compute. But, in my experience, there is no
> > relationship> between the length of the out-of-sample period and
> > the length of the
> > > in-sample period. And gathering the data and performing the
> > calculation of
> > > the ratio for some objective functions would be difficult.
> > >
> > > Isn't it the out-of-sample results we are trying to estimate?
> > Do we care
> > > what the in-sample results look like? If the out-of-sample
> > results are
> > > terrible, why bother computing the ratio. If the out-of-
> > sample results are
> > > good, why bother computing the ratio -- how will that
> > information be used to
> > > improve the system? And if it is used to modify the system,
> > then the
> > > previously out-of-sample data has become in-sample.
> > >
> > > Do any of the forum member have examples they can contribute
> > where computing
> > > the ratio is helpful?
> > >
> > > Thanks,
> > > Howard
> > >
> >
> >
> >
>