|
PureBytes Links
Trading Reference Links
|
>However, the best systems are in the 45% to 55% range so far. What
> does that tell you?
I think you will understand that I have to act from my current biases
but that I am also capable of change - keeping that in mind.
It is meaningless without the frequency distribution of the sample
(it doesn't have to be a normal dist, since the central limit theorem
predicts it will move towards normal behaviour anyway) i.e. W/L
without ave%w/ave%L doesn't tell us as much about outcomes as it does
when combined, let alone simulated.
brian_z
--- In amibroker@xxxxxxxxxxxxxxx, Dennis Brown <see3d@xxx> wrote:
>
> Paul,
>
> Like you said, I always assume that if I start trading a new
system
> that the first thing I will experience is the worst case drawdown
that
> will shake my confidence in it. I have to ask myself how I would
feel
> after a few days of this, and would I continue, or start to
question
> my system development methods and drop out just when the system
turns
> around. This psychology happens to traders and investors all the
> time, and is one of the main causes of losses.
>
> Brian,
>
> I have put together a lot of systems recently, looking for the
most
> robust ones. I have profitable systems with a Win% ranging from
30%
> to 80%. The higher the Win%, the larger the drawdowns. No free
> lunch. However, the best systems are in the 45% to 55% range so
far.
> What does that tell you?
>
> Best regards,
> Dennis
>
> On Mar 13, 2008, at 8:27 PM, Paul Ho wrote:
>
> > Before One can look at OOS, Once needs to have a system or
several
> > systems Candidate and so you still need to have metrics to
evaluate
> > what basically is forward looking performance estimates.
Besides,
> > just because OOS says it is fine, it doesnt mean it will
continue
> > into the future. And you are still more prone to start off at
the
> > flat spot of the equity curve if you have a system that have
more
> > deviation from a straightness of an equity curve regardless of
how
> > much OOS you have done.
> >
> > Your other statements seems more like motherhood statements than
> > looking for the mother ship to me.
> > From: amibroker@xxxxxxxxxxxxxxx
[mailto:amibroker@xxxxxxxxxxxxxxx]
> > On Behalf Of brian_z111
> > Sent: Friday, 14 March 2008 11:03 AM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: [amibroker] Re: What is best statistic for straightness
of
> > equity curve?
> >
> > I agree with Howard's (past) comments that the best metric is the
OOS
> > metric (that is for those who have used optimization to design the
> > system) or better still, several OOS metrics (if we have the
data).
> >
> > The speculative (at this stage) point that I am introducing into
the
> > discussion is that foward looking performance can be estimated
from
> > the root causes (mechanics of the trading system).
> >
> > By continually focussing on the unknown future we are chasing
> > phantasms. While we try to catch one others are popping up
everywhere
> > (just like a horror movie).
> >
> > Better off to find the mother ship, and understand the spawning
> > process, if we are to have any hope of dealing with the offspring.
> >
> > brian_z
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "Paul Ho" <paultsho@> wrote:
> > >
> > > This particular shortcoming of Sharpe ratio as mentioned by
Howard
> > has been
> > > well flaged by many books. and It make sense when one is
comparing
> > PAST
> > > performance from one fund manger to another, or from one system
to
> > another.
> > > However, when one is comparing forward looking performance,
such as
> > when one
> > > is developing new systems or evaulating new variations of an
> > existing
> > > system. Then IMHO this criticism is a little unjustified.
Reason:
> > If there
> > > are an equity curve in front of me, one that is with a
occasional
> > surge of
> > > profit (positive deviation) followed by a relatively flat
patch. I
> > wouldn't
> > > know with a lot of confidence I'm go to experience a flat patch
or
> > > continuing surge if I trade this system in the future. I have
seen
> > a number
> > > of systems that have a very quick rise in patches during
backtest
> > and
> > > optimisation, but basically flat during forward testing. If I
have
> > a choice,
> > > I would prefer a lower return but with less deviation (both
> > positive and
> > > negative) when I'm developing new system because I'm more
confident
> > that it
> > > will generate a regular profit for me. I must confess I am a
short
> > term
> > > trader, my trades last for hours to days. I can apprecriate that
> > long term
> > > traders, those with trades lasting weeks to years, might have a
> > different
> > > psychology and can withstand large period of flat patches to
wait
> > for the
> > > big one. Of course, once I have started using a system, I'm all
for
> > positive
> > > surprises.
> > > I personally think the biggest drawback of Sharpe ratio lies
with
> > the fact
> > > that the straightness of an equity curve cannot be adequately
> > described by a
> > > single Sharpe Ratio, because vastly different equity curves
shares
> > similar
> > > ratio numbers. A series of Sharpe Ratios measured periodically
is a
> > better
> > > guide. Tuschar Chande even went as far as suggesting measuring
> > a "Sharpe
> > > Ratio" over the series of Sharpe Ratio, I think this has merit.
> > >
> > >
> > > _____
> > >
> > > From: amibroker@xxxxxxxxxxxxxxx
[mailto:amibroker@xxxxxxxxxxxxxxx]
> > On Behalf
> > > Of Dennis Brown
> > > Sent: Friday, 14 March 2008 4:46 AM
> > > To: amibroker@xxxxxxxxxxxxxxx
> > > Subject: Re: [amibroker] Re: What is best statistic for
> > straightness of
> > > equity curve?
> > >
> > >
> > >
> > > Howard,
> > >
> > >
> > > You make an excellent point. The metrics used to evaluate a
system
> > needs to
> > > take into consideration the normal "character" of the trading
> > systems basic
> > > methodology.
> > >
> > > For instance my system takes small profits and losses many
times a
> > day. It
> > > is not biased for long or short. It does not hold overnight, It
> > only trades
> > > broad market futures. It does not compound equity. It is
goodness
> > be able
> > > to take a consistent draw from a fixed account size.
> > >
> > > This means that my system will be subject to very different
market
> > forces
> > > than a system that swing trades stocks for a week or two, and is
> > subject to
> > > overnight gaps, company earnings announcements, dividends,
interest
> > rates
> > > (on margin accounts), and other unpredictable events.
> > >
> > > My system will perform with a much smoother equity curve just
> > because of the
> > > way it is defined. Commissions and Bid/Ask spreads are the main
> > hurdles to
> > > profitability, but they are constants.
> > >
> > > I have a much easier time telling if my system is robust.
> > >
> > > Best regards,
> > > Dennis
> > >
> > >
> > > On Mar 13, 2008, at 1:01 PM, Howard B wrote:
> > >
> > >
> > >
> > > Greetings all --
> > >
> > > Professional money managers are sometimes evaluated based on the
> > Sharpe
> > > Ratio of their performance, so it has some value. But, in my
> > research, I
> > > have not found Sharpe Ratio to be a very good metric for use
when
> > developing
> > > systems. Yes, higher Sharpe Ratios will have smaller standard
> > deviations
> > > than lower Sharpe Ratios, but the standard deviation includes
both
> > positive
> > > and negative deviations. That is, it penalizes both positive and
> > negative
> > > performance. If you are designing trend following systems with
> > long holding
> > > periods, and looking for the infrequent large gains associated
with
> > this
> > > type of system, Sharpe Ratio penalizes these. When Sharpe Ratio
is
> > used as
> > > the objective function in an automated walk forward process,
systems
> > > selected as the best in-sample often perform much less well out-
of-
> > sample
> > > than systems selected using K-Ratio, RRR, CAR/MDD, or UPI.
> > >
> > > Thanks for listening,
> > > Howard
> > >
> > >
> > >
> > > On Wed, Mar 12, 2008 at 10:33 PM, Paul Ho <paultsho@xxxxxx
> > > <mailto:paultsho@> com.au> wrote:
> > >
> > >
> > >
> > >
> > > Time doesnt permit me to write a long post. But I think Jack
> > Schwager in one
> > > of his books povides a very good description of what You want.
> > Tuschar
> > > Chande also has insights.
> > > One such parameter is the Sharpe ratio, but you need use it
slightly
> > > differently. Firstly, take risk free return as zero, and you are
> > obtaining
> > > the ratio of mean return to std deviation. Secondly, calculated
> > yearly
> > > sharpe ratios and compare them from year to year.
> > >
> > >
> > > _____
> > >
> > > From: amibroker@xxxxxxxxx <mailto:amibroker@xxxxxxxxxxxxxxx>
ps.com
> > > [mailto:amibroker@xxxxxxxxx <mailto:amibroker@xxxxxxxxxxxxxxx>
> > ps.com] On
> > > Behalf OfDennis Brown
> > > Sent: Thursday, 13 March 2008 12:24 PM
> > > To: amibroker@xxxxxxxxx <mailto:amibroker@xxxxxxxxxxxxxxx>
ps.com
> > > Subject: Re: [amibroker] Re: What is best statistic for
> > straightness of
> > > equity curve?
> > >
> > >
> > >
> > > Brian,
> > >
> > > Thanks for your reply.
> > >
> > > My thinking is that the Std Error will work. I do not need to
use a
> > > Log function on my equity curve, because I do not compound my
> > results,
> > > so they are linear. I also base my work on constant range bars,
so
> > > that linearizes the curves even more. Profit potential can only
> > come
> > > from price movement. The smoothest and straightest equity curves
> > come
> > > from the most robust systems. Period. You can look at the curve
and
> > > judge it, or find a number that is associated with this
property.
> > >
> > > However, step functions get introduced into your nice trading
> > system
> > > from big news events that change the character of the markets
> > > overnight, or in a minute during the day. I consider these
things
> > > that produce large quick drawdowns will be captured by a Maximum
> > > Drawdown metric. The test period needs to have some of these big
> > > events in it. The event may be too quick to affect a large
> > > statistical function much, giving a false sense of goodness to
the
> > > system. Or the perturbation might show up in a way that takes a
> > great
> > > system and makes the smoothness number look bad due to a one
time
> > > event. That is the challenge with a single number, so I will
have
> > to
> > > experiment with the right weightings.
> > >
> > > That is why I say that the absolute judgement comes from
> > examination
> > > of the equity curve. The goodness numbers are just for ease of
> > > relative comparisons of automated parameter optimization for
> > candidate
> > > systems. It is also nice to have a number or two as a future
point
> > of
> > > reference rather than going back over equity curves for every
> > > comparison.
> > >
> > > Perhaps an FFT over the equity curve would generate an
interesting
> > > signature in the period of the dominant frequency and I also
need
> > the
> > > amplitude. I would have to look into this more, since I have not
> > > tried this before.
> > >
> > > I will start out simple and see how better numbers compare to
the
> > > curves, then decide where to go from there.
> > >
> > > > (Why don't you just start posting some of your bits and
pieces,
> > like
> > > > your new PlotShapes PDF, to the UKB - it is a live site - we
don't
> > > > have to wait for the big bang moment to become an author - a
lot
> > of
> > > > my stuff is mundane and/or half finished, but it still has its
> > uses).
> > >
> > > I am buried in work right now, so I wanted to gauge the value to
> > > others of some of the things I could post on the UKB. I would
have
> > to
> > > fight for the time to figure out how to post and fiddle with
with
> > > formatting issues etc. If it were as easy as sending a PDF email
> > > attachment here, I would have done it a month ago. It is the up
> > front
> > > time investment that is holding me back right now.
> > >
> > > When I get little feedback or interest from a post, I can't
> > prioritize
> > > the time to share more of what I am doing. If I were not so
busy, I
> > > would do it anyway, but for now I need powerful justification to
> > delay
> > > some other important work to make time for it. This is not a
spare
> > > time hobby for me, because I have no spare time right now. :-(
> > >
> > > I could use a teammate to get me through the initial stages.
> > However,
> > > I see that only a few have ventured as far as posting yet, so
the
> > > field is limited. I do all my content creation on a Mac, and
keep
> > my
> > > virtual PC free of everything but AmiBroker and related support
> > > programs. That is why I prefer to generate PDF content as it
works
> > > everywhere. And I have exceptionally easy to use and powerful
tools
> > > for generating them already.
> > >
> > > Best regards,
> > > Dennis Brown
> > >
> > > On Mar 12, 2008, at 7:19 PM, brian_z111 wrote:
> > >
> > > > Dennis,
> > > >
> > > > So where is your thinking on this now?
> > > >
> > > >
> > > > (I have been following and I am building to some possible
input
> > but
> > > > since I don't understand logs and barely understand standard
> > error I
> > > > have had to go back to school - it takes quite a while for me
to
> > get
> > > > my head around that stuff and interpret it into trade talk).
> > > >
> > > > I have taken a different approach to evaluation (which is
still a
> > > > work in progress) and based on that I am inclined to the view
that
> > > > evaluations on one equity curve are on rather weak ground -
IMO
> > > > simulation is required for analysis of 'what counts most'.
> > > >
> > > > Also I am zeroing in on the root causes of equity curve
profiles
> > and
> > > > measuring smoothness of a curve is measuring the effect.
> > > >
> > > > BTW - your pane based analysis is very interesting but I think
> > > > ultimately it might prove to have some limitations for good
> > > > evaluation (but not if we correctly identify root causes - we
can
> > > > just pick them out, add some mathematical antecedents and
then we
> > > > will now the answers that simulation will give us and not
need to
> > > > bother the processor - I have convinced myself that this is
in my
> > > > grasps and later I hope the maths people will connect my
> > conceptual
> > > > does and bingo, we are there).
> > > >
> > > > However, I love your question and approach, so over to your
> > immediate
> > > > problem (I had it in mind to go to town on an equity curve
> > smoothness
> > > > metric anyway).
> > > >
> > > > K-ratio is actually a risk reward metric (is that what you
want)?
> > > >
> > > > It also (to me) gets a little mysterious in its workings
(Klestner
> > > > doesn't fully explain one part of it - not from my, lay,
point of
> > > > view anyway).
> > > >
> > > > I am still thinking about it.
> > > >
> > > > So far I would say StDev is out.
> > > > StandardError will do exactly what you say you want to do (as
far
> > as
> > > > I can tell - once again the stats teachers seem to find it
hard to
> > > > put it into trade talk - I see it explained in different ways
in
> > > > different books).
> > > >
> > > > I haven't reached a final conclusion but it seems most likely
> > that if
> > > > you use Standard Error on a compounded equity curve with the
LogN
> > > > approach taken by Klestner you are there - no need to go past
> > that -
> > > > my reservation is based on the fact that I am not sure how to
> > handle
> > > > standardisation - I only work in relative % change - Klestner
> > > > attempts to standardise the K-ratio - he had some trouble
with it
> > to
> > > > start out and had to add a standardising factor.
> > > >
> > > >> Everything I do is in indicator mode in realtime. I build
all my
> > > >> metrics into my AFL. My charts and numbers always match and
all
> > > >> my
> > > >> settings are stored in my Flexible Parameters scheme for
> > different
> > > >> test systems. It is a little different approach, but that is
one
> > > >> of
> > > >> the beauties of AB --that it allows a lot of flexibility of
doing
> > > >> your
> > > >> own thing if you don't want to use the built-in ways.
> > > >
> > > > Yes, all of my evaluation methods are home made, or adaptions
of
> > > > popular methods - works for me.
> > > >
> > > > As I said - if you want all of your evaluation in one window
you
> > > > might need a math formula to sum up the transition from root
> > cause to
> > > > simulation (I naively believe I have the beginning and end in
the
> > bag
> > > > and conceptually the middle formula seems attainable).
> > > >
> > > > (Why don't you just start posting some of your bits and
pieces,
> > like
> > > > your new PlotShapes PDF, to the UKB - it is a live site - we
don't
> > > > have to wait for the big bang moment to become an author - a
lot
> > of
> > > > my stuff is mundane and/or half finished, but it still has its
> > uses).
> > > >
> > > > brian_z
> > > >
> > > >
> > > > --- In amibroker@xxxxxxxxx <mailto:amibroker%
40yahoogroups.com>
> > ps.com,
> > > Dennis Brown <see3d@> wrote:
> > > >>
> > > >> Howard,
> > > >>
> > > >> Thanks for the input. I will investigate these some more.
> > > >>
> > > >> However, I do not use the built-in equity functions, or any
of
> > the
> > > >> built-in trading functions. Tomasz has done a wonderful job
with
> > > >> these, but they do not fit well with what I am doing with my
> > > > trading.
> > > >> I find it easier to understand what I am getting if I write
> > > > everything
> > > >> myself just for my situation and not the general case.
> > > >>
> > > >> Everything I do is in indicator mode in realtime. I build
all my
> > > >> metrics into my AFL. My charts and numbers always match and
all
> > > > my
> > > >> settings are stored in my Flexible Parameters scheme for
> > different
> > > >> test systems. It is a little different approach, but that is
one
> > > > of
> > > >> the beauties of AB --that it allows a lot of flexibility of
doing
> > > > your
> > > >> own thing if you don't want to use the built-in ways.
> > > >>
> > > >> Sometimes, you have to march to the beat of a different
drummer
> > to
> > > >> make money in these markets.
> > > >>
> > > >> Thanks again,
> > > >> Dennis Brown
> > > >>
> > > >>
> > > >> On Mar 12, 2008, at 1:38 PM, Howard B wrote:
> > > >>
> > > >>> Hi Dennis --
> > > >>>
> > > >>> There are several metrics already built in to AmiBroker that
> > > > measure
> > > >>> both the steepness and smoothness of the equity curve. Try
> > > >>> generating a few test runs, plot their equity curves, note
the
> > > >>> values of these metrics, and see which ones best fit your
> > > > trading
> > > >>> personality. A nice advantage to using these is that they
> > > > usually
> > > >>> tend to select trading systems that test well out-of-
sample, so
> > > > are
> > > >>> appropriate for use with the Walk-Forward technique now also
> > > > built
> > > >>> in to AmiBroker.
> > > >>>
> > > >>> KRatio
> > > >>> CAR/MDD
> > > >>> RAR/MDD
> > > >>> RRR
> > > >>> RecoveryFactor
> > > >>> UlcerPerformanceIndex
> > > >>>
> > > >>> Thanks,
> > > >>> Howard
> > > >>>
> > > >>> On Tue, Mar 11, 2008 at 6:06 PM, Dennis Brown <see3d@>
> > > >>> wrote:
> > > >>> Hello,
> > > >>>
> > > >>> I have my system for intraday trading complete enough that I
> > need
> > > > to
> > > >>> start selecting goodness criteria for comparing variations.
I
> > have
> > > >>> selected a number of metrics to display in realtime for an
n day
> > > >>> backtest like:
> > > >>>
> > > >>> total trade count
> > > >>> average bars per trade
> > > >>> winning trade %
> > > >>> trade bars % in green
> > > >>> best trade $
> > > >>> worst trade $
> > > >>> average win $
> > > >>> average loss $
> > > >>> *total profit $
> > > >>> *max draw down $
> > > >>> *EDGE (average $ per trade)
> > > >>> *I have a graph of the cumulative profit over time and an
> > overlaid
> > > >>> straight line plot. This is the most powerful tool, because
it
> > > > lets
> > > >>> me see the real character of the system. The straighter the
> > line,
> > > > the
> > > >>> less likely it is over fit to the data and represents a
robust
> > > > system.
> > > >>>
> > > >>> I also have a graph of the trade equity on a trade by trade
> > > > basis, so
> > > >>> I can see how good the entry timing is and how a trade
> > progresses
> > > > on
> > > >>> average or in outlier conditions.
> > > >>>
> > > >>> The * items are my key metrics for system comparison. This
> > simple
> > > >>> system runs completely in indicator mode. I test about 1000-
2000
> > > >>> trades over a 10 week test period.
> > > >>>
> > > >>> Because of the type and manner of my trades (1 futures
contract
> > > > only
> > > >>> traded during market hours), the data is easy to judge for
> > > > goodness.
> > > >>> Since every day is an island, I could even use interesting
> > random
> > > > day
> > > >>> strategies for in and out of sample data, but so far I just
use
> > > >>> various sequential segments.
> > > >>>
> > > >>> However, when I am spinning my scroll wheel on parameters
while
> > > >>> looking at my charts, it would be nice to have a number that
> > > >>> represents how straight the equity curve is as a first
pass --
> > > >>> especially for when I partially automate the optimization
> > > > process
> > > >>> later.
> > > >>>
> > > >>> I thought I would just take the standard deviation of the
whole
> > > > curve
> > > >>> to the straight line. This is easy. But I think some of you
have
> > > >>> given this problem a lot of thought and I figured one of
you may
> > > > have
> > > >>> some additional insights into the best method for getting a
> > > > meaningful
> > > >>> number for straightness/smoothness of the equity curve. So
here
> > I
> > > > put
> > > >>> the question to you now with an open mind, before I become
set
> > in
> > > > my
> > > >>> ways ;-)
> > > >>>
> > > >>> Best regards,
> > > >>> Dennis Brown
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>
> > > >
> > > >
> > > >
> > > >
> > > > Please note that this group is for discussion between users
only.
> > > >
> > > > To get support from AmiBroker please send an e-mail directly
to
> > > > SUPPORT {at} amibroker.com
> > > >
> > > > For NEW RELEASE ANNOUNCEMENTS and other news always check
DEVLOG:
> > > > http://www.amibroke <http://www.amibroker.com/devlog/>
> > r.com/devlog/
> > > >
> > > > For other support material please check also:
> > > > http://www.amibroke <http://www.amibroker.com/support.html>
> > > r.com/support.html
> > > >
> > > > Yahoo! Groups Links
> > > >
> > > >
> > > >
> > >
> >
> >
> >
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|