|
PureBytes Links
Trading Reference Links
|
That was one of the more interesting and telling posts! Thank you Ivo for
presenting that for all of us. It is a real great lesson. In particular,
your statement:
What I am finding is that this way of testing makes creating a consistently
good system almost impossible!
Really hits the nail on the head in system development. In my quest to find
a good system I've read pretty much read every system trading book from
1980 to today and coded many of them myself. I came to realize a few points
which I believe are confirmed by a number of the more above-board
system-trading review texts. But, I certainly invite different opinions and
information from everyone here.
1) Quality of market: This is often a vague notion perpetrated by system
developers to justify why their "back tested" system works great on one
market but not on another. I find it nonsense. First of all, if remove time
frame and value clues and then look at charts of sunspot activity, wheat,
bonds, housing starts, IBM, even emotional highs and lows; you can not tell
one from another. But, there are a few simple things you can tell: level,
volatility, trendiness, direction. What else is there? And those four are
easily captured by a large number of commonly provided indicators (close,
true range, adx and momentum, respectively for example.) If that is the
case, it seems like any system could test for those four things as a first
filter and then initiate more filters or trades. Sounds simple? Yes. Does
it always work? No.
2) System percentage correct: If you have a system that is correct 30%-50%
of the time, that's not bad! (Heaven forbid a pilot or any other type of
professional has such a low success rate. But, that's part of why it is
psychologically difficult to be a trader.) Of course you could increase
that percentage correct considerably by increasing your risk/stop points.
If you have Larry Williams Iron Discipline and Will, you can have 80%
correct systems. But as he says (paraphrased), "When I'm wrong, I really
want to feel the pain." (Read "large losses here and gut churning
drawdowns" here.) Of course if you are a smart trader using money
management and position sizing, that reduces the number of contracts you
can trade and hence your return.
3) What is a "consistently good system:" Very simply, one with a positive
expectation over time and across markets. Think Las Vegas. Think, "Why are
they building all those million dollar casino's and kids rides with near
free food, drink etc." Think small positive expectation over time and
across markets. The key word is "time." The key meaning is "patience."
Something I, as a newcomer, lack completely and it beats me every time.
4) Optimization: A good mark of over optimization: your system works really
really well, rarely. Works on wheat, sucks on bonds. Or works on bonds but
only between 1990-1995. Or works on the first half of the year but not the
second half. TS's optimization is a wonderful thing and at the same time, a
spawn of the antichrist :) I think your email very correctly points this
out.
5) Psychology: There is one very common thing about all the markets, human
psychology. Other than Drummond Geometry, I've seen very little in the way
of method or system that so comprehensively encapsulates the very thing
that moves markets: people.
Maybe there are great systems out there in the normal view of "great."
Correct most of the time. Any market. Any time frame. Little drawdown. No
series of losing trades greater than 3. Return 50% per trade on average.
Maybe I didn't spend enough money yet? Maybe I should code out 3 more books
worth of systems? Maybe Larry's holding out on me? I don't know.
My challenge to this list: If any of you out there have a profitable system
from actual trades, not back-tested or out of sample blah blah blah, that
works across markets and time frames, let us know. You don't even have to
tell us how it works. Just a few simple statistics would be fine. Resolve
my crisis of faith and restore me in those pure simple baptismal waters of,
"If a bar does this and a another bar does that and an indicator does this
then buy buy buy!" If I find one, I'll be more than happy to let everyone
know. There's enough money moving through the international markets to make
us all very wealthy :)
Final suggestion: Check out Hill and Pruitt's Futures Truth and Babcock's
Commodity Trading Consumer Reports for an eye opening look at what a
"consistently good system" actually means.
Good luck Ivo and thank you again for shining that bright light. Even if it
is in my eyes <:-o)
Thomas "Recently Unemployed with Plenty of Typing Time" Alexander
Boulder, Colorado
"Ivo Karindi"
<ivo@xxxxxxxx To: "OmegaList \(E-mail\)"
m> <omega-list@xxxxxxxxxx>
cc:
02/20/2001 Subject: Behaviour of different contract months
01:59 PM / "Lemon contracts"?
Please
respond to
ivo
I have been wrestling with a system based on an indicator. The personality
of the indicator is such that it will get very "upset" when there are big
jumps in data such as roll-overs in futures contracts. So I decided to
test
the system in a parallel manner on different contracts, not on continuous
data, and combine the results into one single report. Particularly, I test
it on 12 different contracts (intraday) of the same commodity (4 having
overall trend up, 4 down, and 4 sideways).
What I am finding is that this way of testing makes creating a consistently
good system almost impossible! Whatever I try, there always are some
contracts that only have losing trades, and there are always contracts that
have consistently winning ones. But it seems to be almost impossible to
reach a setup where most of the contracts would be profitable ? there
always
are some "lemons".
Now analysing the "lemons", it is interesting that there does not seem to
be
much consistency among them. When optimizing the inputs one way, some of
the contracts are consistent winners and others are consistent losers.
Changing the inputs and optimizing the system for other contract months
these become winners making the previous winners total losers. It seems as
if different contracts of the same commodity have totally different
"personalities", and therefore should be treated completely differently.
However, when applying the same system to a continuous data, it is
relatively easy to turn out a nice profitable equity curve.
The system that I am testing was initially of medium complexity, but in
order to figure out what is happening I have been trying out some very
simple entry/exit rules like entering at x day high and exiting 5 or 10
bars
later. The problem remains the same: some contracts are consistent winners
and some others are consistent losers. Changing inputs, losers and winners
change places but still the overall numbers of winners and losers among
contracts remain the same.
Now what do I have here? Is this a fundamental understanding about the
ease
of over-optimization? Meaning that it is so easy to come up with a good
system if testing and optimizing it only on one set of data, even
continuous
futures' data? Does it show that in reality, if applying it to a different
set of data (future), the system may be totally worthless? Or am I totally
wrong by trying to test a system on different contract months of the same
commodity independently?
One explanation that I may have to the phenomena, and that I have not
tested
out yet, is different demand at different times. The thing might be that,
for example, from Jan to Jun expirations, the contracts may have different
personality than from July to Dec expirations (yes, I work with energies).
This is the next thing I will try out; maybe I will end up with 2 systems ?
one for the first half of the year and the other for the rest. But still
it
is perplexing that such tremendous "behavioural" differences exist among
contracts of the same commodity.
Ivo
|