|
PureBytes Links
Trading Reference Links
|
Hello Kevin,
My maths background is pretty narrow.
Can you give me just a little more so I can follow up?
brian_z
--- In amibroker@xxxxxxxxxxxxxxx, Kevin243@xxx wrote:
>
>
> I like R**2... a bit of programming, but can be done...
>
> Kevin Campbell
>
>
> In a message dated 3/13/2008 7:15:09 PM Central Daylight Time,
> brian_z111@xxx writes:
>
> BTW
>
> That is a slightly different discussion to the one on deciding
which
> ObjectiveFunction is superior (relatively speaking) but if we
don't
> understand the first point what basis do we have for making a
> decision on ObjectiveFunctions?
>
> Where I am at is to dig in further on the mechanics, with the view
to
> checking the value of our current thinking. The eventual OF that
> comes out of that research may well surprize.
>
> brian_z
>
>
> --- In amibroker@xxxxxxxxxxxxxxx, "brian_z111" <brian_z111@> wrote:
> >
> > I agree with Howard's (past) comments that the best metric is
the
> OOS
> > metric (that is for those who have used optimization to design
the
> > system) or better still, several OOS metrics (if we have the
data).
> >
> > The speculative (at this stage) point that I am introducing into
> the
> > discussion is that foward looking performance can be estimated
from
> > the root causes (mechanics of the trading system).
> >
> > By continually focussing on the unknown future we are chasing
> > phantasms. While we try to catch one others are popping up
> everywhere
> > (just like a horror movie).
> >
> > Better off to find the mother ship, and understand the spawning
> > process, if we are to have any hope of dealing with the
offspring.
> >
> > brian_z
> >
> >
> >
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "Paul Ho" <paultsho@> wrote:
> > >
> > > This particular shortcoming of Sharpe ratio as mentioned by
> Howard
> > has been
> > > well flaged by many books. and It make sense when one is
> comparing
> > PAST
> > > performance from one fund manger to another, or from one
system
> to
> > another.
> > > However, when one is comparing forward looking performance,
such
> as
> > when one
> > > is developing new systems or evaulating new variations of an
> > existing
> > > system. Then IMHO this criticism is a little unjustified.
Reason:
> > If there
> > > are an equity curve in front of me, one that is with a
occasional
> > surge of
> > > profit (positive deviation) followed by a relatively flat
patch.
> I
> > wouldn't
> > > know with a lot of confidence I'm go to experience a flat
patch or
> > > continuing surge if I trade this system in the future. I have
> seen
> > a number
> > > of systems that have a very quick rise in patches during
backtest
> > and
> > > optimisation, but basically flat during forward testing. If I
> have
> > a choice,
> > > I would prefer a lower return but with less deviation (both
> > positive and
> > > negative) when I'm developing new system because I'm more
> confident
> > that it
> > > will generate a regular profit for me. I must confess I am a
> short
> > term
> > > trader, my trades last for hours to days. I can apprecriate
that
> > long term
> > > traders, those with trades lasting weeks to years, might have
a
> > different
> > > psychology and can withstand large period of flat patches to
wait
> > for the
> > > big one. Of course, once I have started using a system, I'm
all
> for
> > positive
> > > surprises.
> > > I personally think the biggest drawback of Sharpe ratio lies
with
> > the fact
> > > that the straightness of an equity curve cannot be adequately
> > described by a
> > > single Sharpe Ratio, because vastly different equity curves
> shares
> > similar
> > > ratio numbers. A series of Sharpe Ratios measured periodically
is
> a
> > better
> > > guide. Tuschar Chande even went as far as suggesting measuring
> > a "Sharpe
> > > Ratio" over the series of Sharpe Ratio, I think this has merit.
> > >
> > >
> > > _____
> > >
> > > From: amibroker@xxxxxxxxxxxxxxx
> [mailto:amibroker@xxxxxxxxxxxxxxx]
> > On Behalf
> > > Of Dennis Brown
> > > Sent: Friday, 14 March 2008 4:46 AM
> > > To: amibroker@xxxxxxxxxxxxxxx
> > > Subject: Re: [amibroker] Re: What is best statistic for
> > straightness of
> > > equity curve?
> > >
> > >
> > >
> > > Howard,
> > >
> > >
> > > You make an excellent point. The metrics used to evaluate a
> system
> > needs to
> > > take into consideration the normal "character" of the trading
> > systems basic
> > > methodology.
> > >
> > > For instance my system takes small profits and losses many
times
> a
> > day. It
> > > is not biased for long or short. It does not hold overnight,
It
> > only trades
> > > broad market futures. It does not compound equity. It is
> goodness
> > be able
> > > to take a consistent draw from a fixed account size.
> > >
> > > This means that my system will be subject to very different
> market
> > forces
> > > than a system that swing trades stocks for a week or two, and
is
> > subject to
> > > overnight gaps, company earnings announcements, dividends,
> interest
> > rates
> > > (on margin accounts), and other unpredictable events.
> > >
> > > My system will perform with a much smoother equity curve just
> > because of the
> > > way it is defined. Commissions and Bid/Ask spreads are the
main
> > hurdles to
> > > profitability, but they are constants.
> > >
> > > I have a much easier time telling if my system is robust.
> > >
> > > Best regards,
> > > Dennis
> > >
> > >
> > > On Mar 13, 2008, at 1:01 PM, Howard B wrote:
> > >
> > >
> > >
> > > Greetings all --
> > >
> > > Professional money managers are sometimes evaluated based on
the
> > Sharpe
> > > Ratio of their performance, so it has some value. But, in my
> > research, I
> > > have not found Sharpe Ratio to be a very good metric for use
when
> > developing
> > > systems. Yes, higher Sharpe Ratios will have smaller
standard
> > deviations
> > > than lower Sharpe Ratios, but the standard deviation includes
> both
> > positive
> > > and negative deviations. That is, it penalizes both positive
and
> > negative
> > > performance. If you are designing trend following systems
with
> > long holding
> > > periods, and looking for the infrequent large gains associated
> with
> > this
> > > type of system, Sharpe Ratio penalizes these. When Sharpe
Ratio
> is
> > used as
> > > the objective function in an automated walk forward process,
> systems
> > > selected as the best in-sample often perform much less well
out-
> of-
> > sample
> > > than systems selected using K-Ratio, RRR, CAR/MDD, or UPI.
> > >
> > > Thanks for listening,
> > > Howard
> > >
> > >
> > >
> > > On Wed, Mar 12, 2008 at 10:33 PM, Paul Ho <paultsho@xxxxxx
> > > <mailto:paultsho@> com.au> wrote:
> > >
> > >
> > >
> > >
> > > Time doesnt permit me to write a long post. But I think Jack
> > Schwager in one
> > > of his books povides a very good description of what You want.
> > Tuschar
> > > Chande also has insights.
> > > One such parameter is the Sharpe ratio, but you need use it
> slightly
> > > differently. Firstly, take risk free return as zero, and you
are
> > obtaining
> > > the ratio of mean return to std deviation. Secondly,
calculated
> > yearly
> > > sharpe ratios and compare them from year to year.
> > >
> > >
> > > _____
> > >
> > > From: amibroker@xxxxxxxxx <mailto:amibroker@xxxxxxxxxxxxxxx>
> ps.com
> > > [mailto:amibroker@xxxxxxxxx <mailto:amibroker@xxxxxxxxxxxxxxx>
> > ps.com] On
> > > Behalf OfDennis Brown
> > > Sent: Thursday, 13 March 2008 12:24 PM
> > > To: amibroker@xxxxxxxxx <mailto:amibroker@xxxxxxxxxxxxxxx>
ps.com
> > > Subject: Re: [amibroker] Re: What is best statistic for
> > straightness of
> > > equity curve?
> > >
> > >
> > >
> > > Brian,
> > >
> > > Thanks for your reply.
> > >
> > > My thinking is that the Std Error will work. I do not need to
use
> a
> > > Log function on my equity curve, because I do not compound my
> > results,
> > > so they are linear. I also base my work on constant range
bars,
> so
> > > that linearizes the curves even more. Profit potential can
only
> > come
> > > from price movement. The smoothest and straightest equity
curves
> > come
> > > from the most robust systems. Period. You can look at the
curve
> and
> > > judge it, or find a number that is associated with this
property.
> > >
> > > However, step functions get introduced into your nice trading
> > system
> > > from big news events that change the character of the markets
> > > overnight, or in a minute during the day. I consider these
things
> > > that produce large quick drawdowns will be captured by a
Maximum
> > > Drawdown metric. The test period needs to have some of these
big
> > > events in it. The event may be too quick to affect a large
> > > statistical function much, giving a false sense of goodness to
> the
> > > system. Or the perturbation might show up in a way that takes
a
> > great
> > > system and makes the smoothness number look bad due to a one
time
> > > event. That is the challenge with a single number, so I will
have
> > to
> > > experiment with the right weightings.
> > >
> > > That is why I say that the absolute judgement comes from
> > examination
> > > of the equity curve. The goodness numbers are just for ease of
> > > relative comparisons of automated parameter optimization for
> > candidate
> > > systems. It is also nice to have a number or two as a future
> point
> > of
> > > reference rather than going back over equity curves for every
> > > comparison.
> > >
> > > Perhaps an FFT over the equity curve would generate an
> interesting
> > > signature in the period of the dominant frequency and I also
need
> > the
> > > amplitude. I would have to look into this more, since I have
not
> > > tried this before.
> > >
> > > I will start out simple and see how better numbers compare to
the
> > > curves, then decide where to go from there.
> > >
> > > > (Why don't you just start posting some of your bits and
pieces,
> > like
> > > > your new PlotShapes PDF, to the UKB - it is a live site - we
> don't
> > > > have to wait for the big bang moment to become an author - a
> lot
> > of
> > > > my stuff is mundane and/or half finished, but it still has
its
> > uses).
> > >
> > > I am buried in work right now, so I wanted to gauge the value
to
> > > others of some of the things I could post on the UKB. I would
> have
> > to
> > > fight for the time to figure out how to post and fiddle with
with
> > > formatting issues etc. If it were as easy as sending a PDF
email
> > > attachment here, I would have done it a month ago. It is the
up
> > front
> > > time investment that is holding me back right now.
> > >
> > > When I get little feedback or interest from a post, I can't
> > prioritize
> > > the time to share more of what I am doing. If I were not so
busy,
> I
> > > would do it anyway, but for now I need powerful justification
to
> > delay
> > > some other important work to make time for it. This is not a
> spare
> > > time hobby for me, because I have no spare time right now. :-(
> > >
> > > I could use a teammate to get me through the initial stages.
> > However,
> > > I see that only a few have ventured as far as posting yet, so
the
> > > field is limited. I do all my content creation on a Mac, and
keep
> > my
> > > virtual PC free of everything but AmiBroker and related
support
> > > programs. That is why I prefer to generate PDF content as it
> works
> > > everywhere. And I have exceptionally easy to use and powerful
> tools
> > > for generating them already.
> > >
> > > Best regards,
> > > Dennis Brown
> > >
> > > On Mar 12, 2008, at 7:19 PM, brian_z111 wrote:
> > >
> > > > Dennis,
> > > >
> > > > So where is your thinking on this now?
> > > >
> > > >
> > > > (I have been following and I am building to some possible
input
> > but
> > > > since I don't understand logs and barely understand standard
> > error I
> > > > have had to go back to school - it takes quite a while for
me
> to
> > get
> > > > my head around that stuff and interpret it into trade talk).
> > > >
> > > > I have taken a different approach to evaluation (which is
still
> a
> > > > work in progress) and based on that I am inclined to the
view
> that
> > > > evaluations on one equity curve are on rather weak ground -
IMO
> > > > simulation is required for analysis of 'what counts most'.
> > > >
> > > > Also I am zeroing in on the root causes of equity curve
> profiles
> > and
> > > > measuring smoothness of a curve is measuring the effect.
> > > >
> > > > BTW - your pane based analysis is very interesting but I
think
> > > > ultimately it might prove to have some limitations for good
> > > > evaluation (but not if we correctly identify root causes -
we
> can
> > > > just pick them out, add some mathematical antecedents and
then
> we
> > > > will now the answers that simulation will give us and not
need
> to
> > > > bother the processor - I have convinced myself that this is
in
> my
> > > > grasps and later I hope the maths people will connect my
> > conceptual
> > > > does and bingo, we are there).
> > > >
> > > > However, I love your question and approach, so over to your
> > immediate
> > > > problem (I had it in mind to go to town on an equity curve
> > smoothness
> > > > metric anyway).
> > > >
> > > > K-ratio is actually a risk reward metric (is that what you
> want)?
> > > >
> > > > It also (to me) gets a little mysterious in its workings
> (Klestner
> > > > doesn't fully explain one part of it - not from my, lay,
point
> of
> > > > view anyway).
> > > >
> > > > I am still thinking about it.
> > > >
> > > > So far I would say StDev is out.
> > > > StandardError will do exactly what you say you want to do
(as
> far
> > as
> > > > I can tell - once again the stats teachers seem to find it
hard
> to
> > > > put it into trade talk - I see it explained in different
ways in
> > > > different books).
> > > >
> > > > I haven't reached a final conclusion but it seems most
likely
> > that if
> > > > you use Standard Error on a compounded equity curve with the
> LogN
> > > > approach taken by Klestner you are there - no need to go
past
> > that -
> > > > my reservation is based on the fact that I am not sure how
to
> > handle
> > > > standardisation - I only work in relative % change - Klestner
> > > > attempts to standardise the K-ratio - he had some trouble
with
> it
> > to
> > > > start out and had to add a standardising factor.
> > > >
> > > >> Everything I do is in indicator mode in realtime. I build
all
> my
> > > >> metrics into my AFL. My charts and numbers always match and
all
> > > >> my
> > > >> settings are stored in my Flexible Parameters scheme for
> > different
> > > >> test systems. It is a little different approach, but that
is
> one
> > > >> of
> > > >> the beauties of AB --that it allows a lot of flexibility of
> doing
> > > >> your
> > > >> own thing if you don't want to use the built-in ways.
> > > >
> > > > Yes, all of my evaluation methods are home made, or
adaptions of
> > > > popular methods - works for me.
> > > >
> > > > As I said - if you want all of your evaluation in one window
you
> > > > might need a math formula to sum up the transition from root
> > cause to
> > > > simulation (I naively believe I have the beginning and end
in
> the
> > bag
> > > > and conceptually the middle formula seems attainable).
> > > >
> > > > (Why don't you just start posting some of your bits and
pieces,
> > like
> > > > your new PlotShapes PDF, to the UKB - it is a live site - we
> don't
> > > > have to wait for the big bang moment to become an author - a
> lot
> > of
> > > > my stuff is mundane and/or half finished, but it still has
its
> > uses).
> > > >
> > > > brian_z
> > > >
> > > >
> > > > --- In amibroker@xxxxxxxxx <mailto:amibroker%
40yahoogroups.com>
> > ps.com,
> > > Dennis Brown <see3d@> wrote:
> > > >>
> > > >> Howard,
> > > >>
> > > >> Thanks for the input. I will investigate these some more.
> > > >>
> > > >> However, I do not use the built-in equity functions, or any
of
> > the
> > > >> built-in trading functions. Tomasz has done a wonderful job
> with
> > > >> these, but they do not fit well with what I am doing with my
> > > > trading.
> > > >> I find it easier to understand what I am getting if I write
> > > > everything
> > > >> myself just for my situation and not the general case.
> > > >>
> > > >> Everything I do is in indicator mode in realtime. I build
all
> my
> > > >> metrics into my AFL. My charts and numbers always match and
all
> > > > my
> > > >> settings are stored in my Flexible Parameters scheme for
> > different
> > > >> test systems. It is a little different approach, but that
is
> one
> > > > of
> > > >> the beauties of AB --that it allows a lot of flexibility of
> doing
> > > > your
> > > >> own thing if you don't want to use the built-in ways.
> > > >>
> > > >> Sometimes, you have to march to the beat of a different
> drummer
> > to
> > > >> make money in these markets.
> > > >>
> > > >> Thanks again,
> > > >> Dennis Brown
> > > >>
> > > >>
> > > >> On Mar 12, 2008, at 1:38 PM, Howard B wrote:
> > > >>
> > > >>> Hi Dennis --
> > > >>>
> > > >>> There are several metrics already built in to AmiBroker
that
> > > > measure
> > > >>> both the steepness and smoothness of the equity curve. Try
> > > >>> generating a few test runs, plot their equity curves,
note the
> > > >>> values of these metrics, and see which ones best fit your
> > > > trading
> > > >>> personality. A nice advantage to using these is that they
> > > > usually
> > > >>> tend to select trading systems that test well out-of-
sample,
> so
> > > > are
> > > >>> appropriate for use with the Walk-Forward technique now
also
> > > > built
> > > >>> in to AmiBroker.
> > > >>>
> > > >>> KRatio
> > > >>> CAR/MDD
> > > >>> RAR/MDD
> > > >>> RRR
> > > >>> RecoveryFactor
> > > >>> UlcerPerformanceIndex
> > > >>>
> > > >>> Thanks,
> > > >>> Howard
> > > >>>
> > > >>> On Tue, Mar 11, 2008 at 6:06 PM, Dennis Brown <see3d@>
> > > >>> wrote:
> > > >>> Hello,
> > > >>>
> > > >>> I have my system for intraday trading complete enough that
I
> > need
> > > > to
> > > >>> start selecting goodness criteria for comparing
variations. I
> > have
> > > >>> selected a number of metrics to display in realtime for an
n
> day
> > > >>> backtest like:
> > > >>>
> > > >>> total trade count
> > > >>> average bars per trade
> > > >>> winning trade %
> > > >>> trade bars % in green
> > > >>> best trade $
> > > >>> worst trade $
> > > >>> average win $
> > > >>> average loss $
> > > >>> *total profit $
> > > >>> *max draw down $
> > > >>> *EDGE (average $ per trade)
> > > >>> *I have a graph of the cumulative profit over time and an
> > overlaid
> > > >>> straight line plot. This is the most powerful tool,
because it
> > > > lets
> > > >>> me see the real character of the system. The straighter
the
> > line,
> > > > the
> > > >>> less likely it is over fit to the data and represents a
robust
> > > > system.
> > > >>>
> > > >>> I also have a graph of the trade equity on a trade by trade
> > > > basis, so
> > > >>> I can see how good the entry timing is and how a trade
> > progresses
> > > > on
> > > >>> average or in outlier conditions.
> > > >>>
> > > >>> The * items are my key metrics for system comparison.
This
> > simple
> > > >>> system runs completely in indicator mode. I test about
1000-
> 2000
> > > >>> trades over a 10 week test period.
> > > >>>
> > > >>> Because of the type and manner of my trades (1 futures
> contract
> > > > only
> > > >>> traded during market hours), the data is easy to judge for
> > > > goodness.
> > > >>> Since every day is an island, I could even use interesting
> > random
> > > > day
> > > >>> strategies for in and out of sample data, but so far I
just
> use
> > > >>> various sequential segments.
> > > >>>
> > > >>> However, when I am spinning my scroll wheel on parameters
> while
> > > >>> looking at my charts, it would be nice to have a number
that
> > > >>> represents how straight the equity curve is as a first
pass --
> > > >>> especially for when I partially automate the optimization
> > > > process
> > > >>> later.
> > > >>>
> > > >>> I thought I would just take the standard deviation of the
> whole
> > > > curve
> > > >>> to the straight line. This is easy. But I think some of
you
> have
> > > >>> given this problem a lot of thought and I figured one of
you
> may
> > > > have
> > > >>> some additional insights into the best method for getting
a
> > > > meaningful
> > > >>> number for straightness/smoothness of the equity curve. So
> here
> > I
> > > > put
> > > >>> the question to you now with an open mind, before I become
> set
> > in
> > > > my
> > > >>> ways ;-)
> > > >>>
> > > >>> Best regards,
> > > >>> Dennis Brown
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>
> > > >
> > > >
> > > >
> > > >
> > > > Please note that this group is for discussion between users
> only.
> > > >
> > > > To get support from AmiBroker please send an e-mail directly
to
> > > > SUPPORT {at} amibroker.com
> > > >
> > > > For NEW RELEASE ANNOUNCEMENTS and other news always check
> DEVLOG:
> > > > http://www.amibroke <http://www.amibroker.com/devlog/>
> > r.com/devlog/
> > > >
> > > > For other support material please check also:
> > > > http://www.amibroke <http://www.amibroker.com/support.html>
> > > r.com/support.html
> > > >
> > > > Yahoo! Groups Links
> > > >
> > > >
> > > >
> > >
> >
>
>
>
> ------------------------------------
>
> Please note that this group is for discussion between users only.
>
> To get support from AmiBroker please send an e-mail directly to
> SUPPORT {at} amibroker.com
>
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> For other support material please check also:
> http://www.amibroker.com/support.html
> Yahoo! Groups Links
>
>
>
>
>
>
>
>
>
>
> **************It's Tax Time! Get tips, forms, and advice on AOL
Money &
> Finance. (http://money.aol.com/tax?NCID=aolprf00030000000001)
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|