[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: How to spot over optimizing



PureBytes Links

Trading Reference Links

Herman,
Of course I am not a gambler.
The RSIt nature, the shape of the certain curve helped decision 
making [always ready to stop the trade, if something would go wrong, 
but it did not...] and another basic thought : a 5/5/0 is more 
probable to be successful in the future than a 5/3/2.
Since trading is a real [and not theoretical] thing, after 6/6/0, 
7/7/0 and the upcoming 8/8/0 and since each one was +40% or +50% or 
+60%, with a strict -5% stop loss I need a lot of time to loose these 
profits.
Another additional point for the # of trades. For a slow system, 2 or 
3 trades per year, I would need 10 years to collect 30 confirmed 
samples. I do not think there is any long term trend holding 10 
years. Markets are changing their style more frequently.
On the other side, for a quick CMO5 system you may have the 30 
samples in two months. Is it enough to decide ? Two months may be a 
single trend for a stock. It will be very dangerous to come at the 
end of the trend, with your 30-sample statistics, to apply a system 
profitable for the ...previous 2-month trend. It is quite out of 
trading logic.
It is very difficult to define a # of trades to confirm, it is 
strongly related to the "periodic" behavior of the stock [or of the 
Market] and our tools for spectrum analysis are very poor [and 
perhaps defective, if we talk for FFT analysis.
I do not trust individual analysis statistics, I prefer TTM logic.
My favorite[s] stock will make profits in harmony with the Market 
profits and this RSIt system is highly profitable for the Market.
And it will be, as I suppose.
I do not know for how long but, I will be there to see the probable 
change. Until then, some profits are not the worse choice and you may 
agree.
Dimitris Tsokakis
--- In amibroker@xxxx, "Herman van den Bergen" <psytek@xxxx> wrote:
> Hi Dimitris,
> 
> Yes, you make some very good points. It is clear that the rules 
change and
> are different for different types of systems. Nothing is ever 
simple... For
> example, the average trade duration and whether the system is short-
term
> oscillator based might be factors that change requirements.
> 
> How did you decide, with the RSIt type system, that you had enough
> information (after only five trades) to put money on the line? I 
don't think
> you are a gambler :-) I readily accept that some of my criteria do 
not apply
> to your case but what criteria did you use?
> 
> My questions were prompted by a system I am testing that performs 
too good
> to be true, I have been picking it apart for a couple of weeks now 
and am
> looking for testing criteria I can throw at it. Perhaps I am 
looking for a
> level of certainty that cannot be found in trading...
> 
> Your comments made me add another item to my list: Minimum number 
of trades.
> 
> Thanks Dimitris,
> Best regards,
> Herman.
> 
> 
> -----Original Message-----
> From: dtsokakis [mailto:TSOKAKIS@x...]
> Sent: Sunday, July 14, 2002 2:19 PM
> To: amibroker@xxxx
> Subject: [amibroker] Re: How to spot over optimizing
> 
> 
> Hi Herman,
> Some of your definitions are quite arbitrary for me.
> I will give an example:
> I was studying last December an RSIt system and found a stock with
> almost ideal performance:
> 5trades/5winners/0losers since 1/2000.
> I decided to follow this example.
> Now, 7 months later, we have already 7trades/7winners/0losers and we
> walk steadily to 8/8/0.
> The last short was at 2.30 and now the recent values are 1.75 to 
1.80.
> The 8th trade will be closed below 1.70 or [hopefully] lower, 
because
> of the bearish sentiment, because of the investing fear, because 
many
> share holders give up, because, because,..., we always hear
> some "reasons" to take the wrong decision...
> Is there any reason to change this trading system because it is
> against
> #1, [yes, it is one stock]
> #4, [yes, a +3500% is too good to be true in 30 ultra bearish 
months]
> #6, [I did not even look at the before 1/2000 extra bullish [and
> catastrophic] period]
> #9 [yes, 5/5/0 changed to 7/7/0 and it will be 8/8/0 in the next two
> weeks]
> and maybe some other statements you refer ?
> What would you do in my position ?
> I would appreciate a reply, if we discuss for real trading and not
> academically.
> Thank you for your interesting, as usual, mail.
> Dimitris Tsokakis
> --- In amibroker@xxxx, "Herman van den Bergen" <psytek@xxxx> wrote:
> > Optimizing has been discussed at several times on this list, but
> never have
> > I seen a check-list that could help you identify over optimizing. 
I
> welcome
> > additions to the small list below. If we can identify degrees of
> > over-optimization we can use the same tool to rate robustness, 
this
> because
> > they are somewhat complementary.
> >
> > If any of the following statements are True you may be over-
> optimizing:
> >
> > 1) The system works on only one stock
> > 2) The gains are erratic with a final big gain
> > 3) Optimization values are scattered (random peaks on a 3D chart)
> > 4) Gains are too good to be true
> > 5) The equity line is flat
> > 6) The system only works in recent years
> > 7) It doesn't work out-of-sample (I use 8y dev, 2y out of sample
> testing)
> > 8) Optimizing parameters vary widely for different time periods
> > 9) Greater than 75% winning trades
> > 7) ... what can you add?
> >
> > It seem that if you overlay the equities for many systems many
> appear to
> > fail for the same periods (use 10 year data), I take this as a 
sign
> that the
> > system pick up on a true market anomaly. It seems to indicates 
that
> there
> > are times that the market is incoherent (chaos rules) and that 
most
> systems
> > will not perform well. Combining equities can produce a "Market
> Coherence"
> > (please suggest a better name) indicator which would tell you when
> trading
> > is highly risky. Or taking the inverse; a "Market Chaos" 
indicator.
> Brave
> > traders will flip all their signals during those periods :-)
> >
> > Since there are many parameters one can optimize I take note of 
the
> gain
> > improvement that comes with selectively activating various stages
> of the
> > system. I start with the most basic form and add/measure features
> as I add
> > them. An unreasonable large jump in gain for any one change makes
> it suspect
> > and I will pick it apart manually for a few trades.
> 
> 
> 
> 
> 
> Your use of Yahoo! Groups is subject to 
http://docs.yahoo.com/info/terms/