|
PureBytes Links
Trading Reference Links
|
Fred,
I haven't studied your I/O work, or any of your other work in detail
as yet, but I would wager a bet that new traders could gain more
insight into I/O by *workshopping* your stuff than they can get from
90% of the books and websites etc that are out there.
This is pretty much true of AmiBroker across the board.
That's a fact that needs to be stated in my book and I am just saying
that for you and Tomasz so you both don't have to grapple with modesty.
*****************************************************************
OOS/WF as a peep into the future?
I'm sorry Fred but OOS/WF does not constitute a peep into the future.
Last night I debated with myself whether I would rebut your arguments
this time around or not and my conclusion was you that would not want
me to humour you.
I will use OOS as the example, as WF is a little more complex with
it's multiple, overlapping sample approach.
OOS is just one sample, no more and no less.
It may be the second one taken but it is still one sample and one
sample only.
The point is, that as a look into the future, it is the inferred
behaviour of the (sampling distributed)population that is important
and one OOS sample a distributed population does not make.
I am not suggesting that we don't do it, or that I don't do it, but
the simple fact is that it is a comfort blanket.
When we look into the future we can't see a clear picture, we can only
see a miasma of probabilities, and we just can not accept that.
It is foreign to our norms and we resist this by denial and looking to
find a real time certainty that we can hang our hats on.
I say that uncertainty is quite embraceable if her features can be
vaguely discerned.
OOS is a not a peep into the future because it is a test sample based
on historical data and it provides only one equity curve based on that
past history.
Now I don't know about you, but the positive equity curve with a high
growth rate is the reason I am playing this game (lucky for me it is
good fun at the same time).
Now the chances of that one OOS equity curve being the exact one
received the first time the system is traded are very slim, so why is
it so important?
Well, it's not.
If any large data sample of trades is walked through using random
selection there is an infinite number of possible equity curves, any
one of which could be *the one* that the trader gets the first time
they trade the system.
It is these equity curves that constitute the range of all possible
trading futures.
Which one we are going to get we don't know and never will, but we can
know our chances of getting any one of them,especially the bad ones.
I will warrant that traders remember their extreme losses more vividly
than their extreme wins.
When enough trade simulations have been conducted, all the equity
curves will fall into a recogniseable distribution; one that
approaches a normal curve.
Unless a trader designs a system that never loses, the equity curve
distributions will include some minus 4 or 5 standard deviation cases
with a probability for each range.
It doesn't matter if that ruinous curve has a 1/1 billion chance of
occurring, it can still occur at anytime,even on the first trading
excursion with the system i.e. during the OOS test.
For that reason I say that in some cases we throw out perfectly good
trading systems because wild chance sometimes throws up a poor
outcome, from a good system, during our OOS test.
The chances of this happening in the OOS test are exactly the same as
happening in the first or any other real time trading scenario.
If we have do one OOS test and it gets our confidence up, should we
then do another one, and another one?
Then we could really start trading the system with confidence>
The trading commentators I have read don't offer any firm criteria for
deciding if a OOS test is acceptable or not.
They generally give a wishy-washy definition e.g. *If the OOOS test is
within 50% of the back-test results*.
Why 50% and what does that mean anyway?
My claim is that we can be definite about it.
If fact we can be just as definite as the margin of error provided by
our sampling technique and subsequent statistical evaluation.
In another post I will provide a simple MCS example, that to my mind
demonstrates this quite clearly.
Anyone still in doubt after that should wait for the *piece de
resistance*, the Random Walk, which ties it all together irrefutably
for anyone who is basing their decisions on pure logic.
BrianB2
--- In amibroker@xxxxxxxxxxxxxxx, "Fred" <ftonetti@xxx> wrote:
>
> OOS and/or WF Testing is not a concept invented in or for IO so that
> particular piece of software is not really the issue per se except
that
> IO has facilitated making it considerably easier to perform.
>
> "If one OOS test had a 50% drawdown it doesn't say that much about
the
> system. It only says something about that one single OOS test of
> however many samples."
>
> It doesn't ? ... It speaks volumes to me ... From my perspective,
> decent in sample performance, whether or not one applies MCS after
the
> fact, is not the end of system testing, it is only a milestone along
> the way to developing a system that MIGHT be tradable ...
>
Content-Description: "AVG certification"
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.409 / Virus Database: 268.14.11/542 - Release Date: 11/20/2006
|