Re: [amibroker] Re: Expectancy - and related--specifically K-rato, AmiBroker Email List Archive

2008/4/16, Louis Préfontaine <rockprog80@xxxxxxxxx>:

Thanks Tomasz!

2008/4/16, Tomasz Janeczko <groups@xxxxxxxxxxxxx>:

> So.. If we consider that walk-forwarding would eliminate all curve-fitting, then what do you consider to be the best variable to walk-forward? RAR, CAR, Net profit%?

The best is your custom metric that fits your trading style.

As far as "generic", most simple and built-in metrics are considered I would opt for CAR/MDD or RAR/MDD

General rule is that your metrics should always include drawdown (risk) measure (in addition to profit measure).

Best regards,
Tomasz Janeczko
amibroker.com

----- Original Message -----

From: Louis Préfontaine

To: amibroker@xxxxxxxxxxxxxxx

Sent: Wednesday, April 16, 2008 6:31 PM

Subject: Re: [amibroker] Re: Expectancy - and related--specifically K-rato

Hi,

Thanks for your answers. I am reading Howard's book right now (approx. half the book remaining) and my concern is really for walk-forward as it seems better than simple optimizing.

In AB 5.08 I can choose the variables to optimize. I tend to like k-ratio because it shows consistency in the results; however the best k-ratios are almost never the best net profit or CAR%. What I have read is here: http://www.addictfx.biz/15-categorie-90719.html (It's in french... sorry for those who can't read french).

So.. If we consider that walk-forwarding would eliminate all curve-fitting, then what do you consider to be the best variable to walk-forward? RAR, CAR, Net profit%?

Thanks,

Louis

2008/4/16, Tomasz Janeczko <groups@xxxxxxxxxxxxx>:

"I've read somewhere" - well the world (and Internet specifically) is full of misinformation.

You really need to read Howard's book. Regular optimization is NOT the same as walk forward.

Walk Forward process actually prevents/minimises curve fitting.

Best regards,
Tomasz Janeczko
amibroker.com

----- Original Message -----

From: Louis Préfontaine

To: amibroker@xxxxxxxxxxxxxxx

Sent: Wednesday, April 16, 2008 5:39 PM

Subject: Re: [amibroker] Re: Expectancy - and related--specifically K-rato

Hi,

I've read somewhere that optimizing (or walk-forwarding) using measures as k-ratio, RRR or max drawdown can lead to curve-fitting and is not a good strategy. Do you agree?

What do you think is the best optimizing strategy?

Thanks,

Louis

2008/4/13, gerryjoz <geraldj@xxxxxxxxxx>:

Grant,
in your post you asked me to elaborate on why i thought the K-ratio
was a waste of space and RRR was simpler/better. What i have found is
that k-ratio is generally lower the higher the exposure for the same
or similar trading systems in back test. If you want a high k-ratio,
according to the AB calc, don't buy or sell!
Here is a contrived (curve-fit) example (run on real data) over a few
years
CAR 33%
Profit factor 7
CAR/MDD 2.8
Max Sys DD % 11.5%
RRR 2.15
K-ratio .096
exposure 49%
#trades 170

the K-ratio definitio in AB help is
"
K-Ratio - Detects inconsistency in returns. Should be 1.0 or more. The
higher K ratio is the more consistent return you may expect from the
system. Linear regression slope of equity line multiplied by square
root of sum of squared deviations of bar number divided by standard
error of equity line multiplied by square root of number of bars. More
information: Stocks & Commodities V14:3 (115-118): Measuring System
Performance by Lars N. Kestner
"
personally i prefer measures which are more easily comprehended. This
one isn't, even tho 40 years ago i did do maths & stats at uni.
In any case, back in May 2004 Tomasz changed the calc...
======>

K-ratio calculation changed. following the change made by its creator,
Mr. Lars Kestner.

Quoting from the book "Quantitative Trading Strategies" from 2003 by
Lars Kestner:

[ - - - ]
" The K-ratio is a unitless measure of performance that can be
compared across markets and time periods. [ - - - ] Traders should
search for strategies yielding K-ratios greater than +0.50. Together,
the Sharpe ratio and K-ratio are the most important
measures when evaluating trading strategy performance. Note: When I
created the K-ratio in 1996, I thought I had created a
robust measure to evaluate performance. In mid-2000, trader Bob Fuchs
brought a small error to my attention regarding the
scaling of the K-ratio. He was correct in his critique and I have
corrected the error in this text. Publications prior to 2002 will
show a different formula for the K-ratio. The updated formula in this
book is correct."

Mr Lars Kestner has corrected his formula based on this critique:
K-ratio = slope / ( sterr * per )

slope: Linear regression slope of equity line
sterr: Standard error of slope
per: Number of periods in the performance test

Special thanks to Jeremy Berkovits who brought that to my attention.

<======
There was quite a bit of discussion at the time.
I understand RRR intuitively, and when i look at the other ratios i
can see why one is higher or lower (with a bit of checking).

Is it possible that there was a typo in the K-ratio correction?
Perhaps Mr Kestner has made another change?
I don't have his books or articles, i just gave up on the k-ratio
because i didn't think it was telling me anything useful.

I would be interested if you or anyone else have run some examples
where K-ratio is high and exposure is high, and what are the other
backtest numbers.

regards
Gerry