|
PureBytes Links
Trading Reference Links
|
Hi Dimitris,
Yes, you make some very good points. It is clear that the rules change and
are different for different types of systems. Nothing is ever simple... For
example, the average trade duration and whether the system is short-term
oscillator based might be factors that change requirements.
How did you decide, with the RSIt type system, that you had enough
information (after only five trades) to put money on the line? I don't think
you are a gambler :-) I readily accept that some of my criteria do not apply
to your case but what criteria did you use?
My questions were prompted by a system I am testing that performs too good
to be true, I have been picking it apart for a couple of weeks now and am
looking for testing criteria I can throw at it. Perhaps I am looking for a
level of certainty that cannot be found in trading...
Your comments made me add another item to my list: Minimum number of trades.
Thanks Dimitris,
Best regards,
Herman.
-----Original Message-----
From: dtsokakis [mailto:TSOKAKIS@x...]
Sent: Sunday, July 14, 2002 2:19 PM
To: amibroker@xxxxxxxxxxxxxxx
Subject: [amibroker] Re: How to spot over optimizing
Hi Herman,
Some of your definitions are quite arbitrary for me.
I will give an example:
I was studying last December an RSIt system and found a stock with
almost ideal performance:
5trades/5winners/0losers since 1/2000.
I decided to follow this example.
Now, 7 months later, we have already 7trades/7winners/0losers and we
walk steadily to 8/8/0.
The last short was at 2.30 and now the recent values are 1.75 to 1.80.
The 8th trade will be closed below 1.70 or [hopefully] lower, because
of the bearish sentiment, because of the investing fear, because many
share holders give up, because, because,..., we always hear
some "reasons" to take the wrong decision...
Is there any reason to change this trading system because it is
against
#1, [yes, it is one stock]
#4, [yes, a +3500% is too good to be true in 30 ultra bearish months]
#6, [I did not even look at the before 1/2000 extra bullish [and
catastrophic] period]
#9 [yes, 5/5/0 changed to 7/7/0 and it will be 8/8/0 in the next two
weeks]
and maybe some other statements you refer ?
What would you do in my position ?
I would appreciate a reply, if we discuss for real trading and not
academically.
Thank you for your interesting, as usual, mail.
Dimitris Tsokakis
--- In amibroker@xxxx, "Herman van den Bergen" <psytek@xxxx> wrote:
> Optimizing has been discussed at several times on this list, but
never have
> I seen a check-list that could help you identify over optimizing. I
welcome
> additions to the small list below. If we can identify degrees of
> over-optimization we can use the same tool to rate robustness, this
because
> they are somewhat complementary.
>
> If any of the following statements are True you may be over-
optimizing:
>
> 1) The system works on only one stock
> 2) The gains are erratic with a final big gain
> 3) Optimization values are scattered (random peaks on a 3D chart)
> 4) Gains are too good to be true
> 5) The equity line is flat
> 6) The system only works in recent years
> 7) It doesn't work out-of-sample (I use 8y dev, 2y out of sample
testing)
> 8) Optimizing parameters vary widely for different time periods
> 9) Greater than 75% winning trades
> 7) ... what can you add?
>
> It seem that if you overlay the equities for many systems many
appear to
> fail for the same periods (use 10 year data), I take this as a sign
that the
> system pick up on a true market anomaly. It seems to indicates that
there
> are times that the market is incoherent (chaos rules) and that most
systems
> will not perform well. Combining equities can produce a "Market
Coherence"
> (please suggest a better name) indicator which would tell you when
trading
> is highly risky. Or taking the inverse; a "Market Chaos" indicator.
Brave
> traders will flip all their signals during those periods :-)
>
> Since there are many parameters one can optimize I take note of the
gain
> improvement that comes with selectively activating various stages
of the
> system. I start with the most basic form and add/measure features
as I add
> them. An unreasonable large jump in gain for any one change makes
it suspect
> and I will pick it apart manually for a few trades.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|