Behaviour of different contract months / "Lemon contracts"?

["Ivo Karindi" <ivo@xxxxxxxxx>]

I have been wrestling with a system based on an indicator.  The personality
of the indicator is such that it will get very “upset” when there are big
jumps in data such as roll-overs in futures contracts.  So I decided to test
the system in a parallel manner on different contracts, not on continuous
data, and combine the results into one single report.  Particularly, I test
it on 12 different contracts (intraday) of the same commodity (4 having
overall trend up, 4 down, and 4 sideways).

What I am finding is that this way of testing makes creating a consistently
good system almost impossible!  Whatever I try, there always are some
contracts that only have losing trades, and there are always contracts that
have consistently winning ones.  But it seems to be almost impossible to
reach a setup where most of the contracts would be profitable – there always
are some “lemons”.

Now analysing the “lemons”, it is interesting that there does not seem to be
much consistency among them.  When optimizing the inputs one way, some of
the contracts are consistent winners and others are consistent losers.
Changing the inputs and optimizing the system for other contract months
these become winners making the previous winners total losers.  It seems as
if different contracts of the same commodity have totally different
“personalities”, and therefore should be treated completely differently.
However, when applying the same system to a continuous data, it is
relatively easy to turn out a nice profitable equity curve.

The system that I am testing was initially of medium complexity, but in
order to figure out what is happening I have been trying out some very
simple entry/exit rules like entering at x day high and exiting 5 or 10 bars
later.  The problem remains the same: some contracts are consistent winners
and some others are consistent losers.  Changing inputs, losers and winners
change places but still the overall numbers of winners and losers among
contracts remain the same.

Now what do I have here?  Is this a fundamental understanding about the ease
of over-optimization?  Meaning that it is so easy to come up with a good
system if testing and optimizing it only on one set of data, even continuous
futures’ data?  Does it show that in reality, if applying it to a different
set of data (future), the system may be totally worthless?  Or am I totally
wrong by trying to test a system on different contract months of the same
commodity independently?

One explanation that I may have to the phenomena, and that I have not tested
out yet, is different demand at different times.  The thing might be that,
for example, from Jan to Jun expirations, the contracts may have different
personality than from July to Dec expirations (yes, I work with energies).
This is the next thing I will try out; maybe I will end up with 2 systems –
one for the first half of the year and the other for the rest.  But still it
is perplexing that such tremendous “behavioural” differences exist among
contracts of the same commodity.

Ivo