PureBytes Links
Trading Reference Links
|
Greetings all --
Professional money managers are sometimes evaluated based on the Sharpe Ratio of their performance, so it has some value. But, in my research, I have not found Sharpe Ratio to be a very good metric for use when developing systems. Yes, higher Sharpe Ratios will have smaller standard deviations than lower Sharpe Ratios, but the standard deviation includes both positive and negative deviations. That is, it penalizes both positive and negative performance. If you are designing trend following systems with long holding periods, and looking for the infrequent large gains associated with this type of system, Sharpe Ratio penalizes these. When Sharpe Ratio is used as the objective function in an automated walk forward process, systems selected as the best in-sample often perform much less well out-of-sample than systems selected using K-Ratio, RRR, CAR/MDD, or UPI.
Thanks for listening, Howard
On Wed, Mar 12, 2008 at 10:33 PM, Paul Ho < paultsho@xxxxxxxxxxxx> wrote:
Time doesnt permit me to write a long post. But I think
Jack Schwager in one of his books povides a very good description of what You
want. Tuschar Chande also has insights.
One such parameter is the Sharpe ratio, but you need
use it slightly differently. Firstly, take risk free return as zero, and you are
obtaining the ratio of mean return to std deviation. Secondly, calculated yearly
sharpe ratios and compare them from year to
year.
Brian,
Thanks for your reply.
My thinking is that the Std
Error will work. I do not need to use a Log function on my equity curve,
because I do not compound my results, so they are linear. I also base my
work on constant range bars, so that linearizes the curves even more.
Profit potential can only come from price movement. The smoothest and
straightest equity curves come from the most robust systems. Period. You
can look at the curve and judge it, or find a number that is associated
with this property.
However, step functions get introduced into your
nice trading system from big news events that change the character of the
markets overnight, or in a minute during the day. I consider these things
that produce large quick drawdowns will be captured by a Maximum
Drawdown metric. The test period needs to have some of these big
events in it. The event may be too quick to affect a large statistical
function much, giving a false sense of goodness to the system. Or the
perturbation might show up in a way that takes a great system and makes
the smoothness number look bad due to a one time event. That is the
challenge with a single number, so I will have to experiment with the
right weightings.
That is why I say that the absolute judgement comes
from examination of the equity curve. The goodness numbers are just for
ease of relative comparisons of automated parameter optimization for
candidate systems. It is also nice to have a number or two as a future
point of reference rather than going back over equity curves for every
comparison.
Perhaps an FFT over the equity curve would generate an
interesting signature in the period of the dominant frequency and I also
need the amplitude. I would have to look into this more, since I have not
tried this before.
I will start out simple and see how better
numbers compare to the curves, then decide where to go from
there.
> (Why don't you just start posting some of your bits and
pieces, like > your new PlotShapes PDF, to the UKB - it is a live site -
we don't > have to wait for the big bang moment to become an author - a
lot of > my stuff is mundane and/or half finished, but it still has its
uses).
I am buried in work right now, so I wanted to gauge the value to
others of some of the things I could post on the UKB. I would have to
fight for the time to figure out how to post and fiddle with with
formatting issues etc. If it were as easy as sending a PDF email
attachment here, I would have done it a month ago. It is the up front
time investment that is holding me back right now.
When I get
little feedback or interest from a post, I can't prioritize the time to
share more of what I am doing. If I were not so busy, I would do it
anyway, but for now I need powerful justification to delay some other
important work to make time for it. This is not a spare time hobby for me,
because I have no spare time right now. :-(
I could use a teammate to
get me through the initial stages. However, I see that only a few have
ventured as far as posting yet, so the field is limited. I do all my
content creation on a Mac, and keep my virtual PC free of everything but
AmiBroker and related support programs. That is why I prefer to generate
PDF content as it works everywhere. And I have exceptionally easy to use
and powerful tools for generating them already.
Best
regards, Dennis Brown
On Mar 12, 2008, at 7:19 PM, brian_z111
wrote:
> Dennis, > > So where is your thinking on this
now? > > > (I have been following and I am building to some
possible input but > since I don't understand logs and barely understand
standard error I > have had to go back to school - it takes quite a
while for me to get > my head around that stuff and interpret it into
trade talk). > > I have taken a different approach to evaluation
(which is still a > work in progress) and based on that I am inclined to
the view that > evaluations on one equity curve are on rather weak
ground - IMO > simulation is required for analysis of 'what counts
most'. > > Also I am zeroing in on the root causes of equity curve
profiles and > measuring smoothness of a curve is measuring the
effect. > > BTW - your pane based analysis is very interesting but
I think > ultimately it might prove to have some limitations for
good > evaluation (but not if we correctly identify root causes - we
can > just pick them out, add some mathematical antecedents and then
we > will now the answers that simulation will give us and not need
to > bother the processor - I have convinced myself that this is in
my > grasps and later I hope the maths people will connect my
conceptual > does and bingo, we are there). > > However, I
love your question and approach, so over to your immediate > problem (I
had it in mind to go to town on an equity curve smoothness > metric
anyway). > > K-ratio is actually a risk reward metric (is that
what you want)? > > It also (to me) gets a little mysterious in
its workings (Klestner > doesn't fully explain one part of it - not from
my, lay, point of > view anyway). > > I am still thinking
about it. > > So far I would say StDev is out. >
StandardError will do exactly what you say you want to do (as far as > I
can tell - once again the stats teachers seem to find it hard to > put
it into trade talk - I see it explained in different ways in > different
books). > > I haven't reached a final conclusion but it seems most
likely that if > you use Standard Error on a compounded equity curve
with the LogN > approach taken by Klestner you are there - no need to go
past that - > my reservation is based on the fact that I am not sure how
to handle > standardisation - I only work in relative % change -
Klestner > attempts to standardise the K-ratio - he had some trouble
with it to > start out and had to add a standardising
factor. > >> Everything I do is in indicator mode in realtime.
I build all my >> metrics into my AFL. My charts and numbers always
match and all >> my >> settings are stored in my Flexible
Parameters scheme for different >> test systems. It is a little
different approach, but that is one >> of >> the beauties of
AB --that it allows a lot of flexibility of doing >> your >>
own thing if you don't want to use the built-in ways. > > Yes, all
of my evaluation methods are home made, or adaptions of > popular
methods - works for me. > > As I said - if you want all of your
evaluation in one window you > might need a math formula to sum up the
transition from root cause to > simulation (I naively believe I have the
beginning and end in the bag > and conceptually the middle formula seems
attainable). > > (Why don't you just start posting some of your
bits and pieces, like > your new PlotShapes PDF, to the UKB - it is a
live site - we don't > have to wait for the big bang moment to become an
author - a lot of > my stuff is mundane and/or half finished, but it
still has its uses). > > brian_z > > > --- In amibroker@xxxxxxxxxxxxxxx,
Dennis Brown <see3d@xxx> wrote: >> >>
Howard, >> >> Thanks for the input. I will investigate these
some more. >> >> However, I do not use the built-in equity
functions, or any of the >> built-in trading functions. Tomasz has
done a wonderful job with >> these, but they do not fit well with
what I am doing with my > trading. >> I find it easier to
understand what I am getting if I write > everything >> myself
just for my situation and not the general case. >> >>
Everything I do is in indicator mode in realtime. I build all my >>
metrics into my AFL. My charts and numbers always match and all >
my >> settings are stored in my Flexible Parameters scheme for
different >> test systems. It is a little different approach, but
that is one > of >> the beauties of AB --that it allows a lot
of flexibility of doing > your >> own thing if you don't want
to use the built-in ways. >> >> Sometimes, you have to march
to the beat of a different drummer to >> make money in these
markets. >> >> Thanks again, >> Dennis
Brown >> >> >> On Mar 12, 2008, at 1:38 PM, Howard
B wrote: >> >>> Hi Dennis
-- >>> >>> There are several metrics already built in
to AmiBroker that > measure >>> both the steepness and
smoothness of the equity curve. Try >>> generating a few test
runs, plot their equity curves, note the >>> values of these
metrics, and see which ones best fit your > trading >>>
personality. A nice advantage to using these is that they >
usually >>> tend to select trading systems that test well
out-of-sample, so > are >>> appropriate for use with the
Walk-Forward technique now also > built >>> in to
AmiBroker. >>> >>> KRatio >>>
CAR/MDD >>> RAR/MDD >>> RRR >>>
RecoveryFactor >>>
UlcerPerformanceIndex >>> >>>
Thanks, >>> Howard >>> >>> On Tue, Mar 11,
2008 at 6:06 PM, Dennis Brown <see3d@xxx> >>>
wrote: >>> Hello, >>> >>> I have my system
for intraday trading complete enough that I need > to >>>
start selecting goodness criteria for comparing variations. I
have >>> selected a number of metrics to display in realtime for
an n day >>> backtest like: >>> >>> total
trade count >>> average bars per trade >>> winning
trade % >>> trade bars % in green >>> best trade
$ >>> worst trade $ >>> average win $ >>>
average loss $ >>> *total profit $ >>> *max draw down
$ >>> *EDGE (average $ per trade) >>> *I have a graph
of the cumulative profit over time and an overlaid >>> straight
line plot. This is the most powerful tool, because it >
lets >>> me see the real character of the system. The straighter
the line, > the >>> less likely it is over fit to the data
and represents a robust > system. >>> >>> I also
have a graph of the trade equity on a trade by trade > basis,
so >>> I can see how good the entry timing is and how a trade
progresses > on >>> average or in outlier
conditions. >>> >>> The * items are my key metrics for
system comparison. This simple >>> system runs completely in
indicator mode. I test about 1000-2000 >>> trades over a 10 week
test period. >>> >>> Because of the type and manner of
my trades (1 futures contract > only >>> traded during
market hours), the data is easy to judge for > goodness. >>>
Since every day is an island, I could even use interesting random >
day >>> strategies for in and out of sample data, but so far I
just use >>> various sequential
segments. >>> >>> However, when I am spinning my
scroll wheel on parameters while >>> looking at my charts, it
would be nice to have a number that >>> represents how straight
the equity curve is as a first pass -- >>> especially for when I
partially automate the optimization > process >>>
later. >>> >>> I thought I would just take the
standard deviation of the whole > curve >>> to the straight
line. This is easy. But I think some of you have >>> given this
problem a lot of thought and I figured one of you may >
have >>> some additional insights into the best method for getting
a > meaningful >>> number for straightness/smoothness
of the equity curve. So here I > put >>> the question to you
now with an open mind, before I become set in > my >>> ways
;-) >>> >>> Best regards, >>> Dennis
Brown >>> >>> >>> >>> >> > > > > >
Please note that this group is for discussion between users
only. > > To get support from AmiBroker please send an e-mail
directly to > SUPPORT {at} amibroker.com > > For NEW RELEASE
ANNOUNCEMENTS and other news always check DEVLOG: > http://www.amibroker.com/devlog/ > >
For other support material please check also: > http://www.amibroker.com/support.html > >
Yahoo! Groups Links > > >
__._,_.___
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
__,_._,___
|