[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: What is best statistic for straightness of equity curve?



PureBytes Links

Trading Reference Links

Dennnis,

What you are telling me is that you have a very good system(s), based 
on the fact that you have been around for a while and that you have 
made honest and earnest enquiries into trading methods (we all end up 
in the same place - if we subject our methods to honest and 
persistent evaluation).

So, you are good at system design.

However, we still have to have objective (and therefore portable) 
methods to evaluate systems (hence our discussion and reference to 
the shared ideas of books).

I am totally with Howard in his efforts to define the best methods of 
objective evaluation.

I don't think you are redefining the genre, or that you are exempt 
from the need to evaluate, or the rules of evaluation.

More so. you are looking at it from a particular perspective and/or 
focussing on one aspect (definitely worth the effort - we might 
squeeze a bit more juice from the orange).

I agree with your, implied, point that it is highly likely that we 
can circumvent a lot of evaluation blather by simply looking at some 
key indicators that quickly summarise where we are at e.g. my recent 
post on SampleError/WinLoss variance, and its application to 
estimating significance, proves that we can do that.

I'll stick my neck out and say nothing is going to break the laws of 
the mechanics of trading but we can go further in explaining it 
better and making it more accessible to others.


brian_z






--- In amibroker@xxxxxxxxxxxxxxx, Dennis Brown <see3d@xxx> wrote:
>
> Howard,
> 
> You make an excellent point.  The metrics used to evaluate a 
system  
> needs to take into consideration the normal "character" of the 
trading  
> systems basic methodology.
> 
> For instance my system takes small profits and losses many times a  
> day.  It is not biased for long or short.  It does not hold 
overnight,  
> It only trades broad market futures.  It does not compound equity.  
It  
> is goodness be able to take a consistent draw from a fixed account 
size.
> 
> This means that my system will be subject to very different market  
> forces than a system that swing trades stocks for a week or two, 
and  
> is subject to overnight gaps, company earnings announcements,  
> dividends, interest rates (on margin accounts), and other  
> unpredictable events.
> 
> My system will perform with a much smoother equity curve just 
because  
> of the way it is defined.  Commissions and Bid/Ask spreads are the  
> main hurdles to profitability, but they are constants.
> 
> I have a much easier time telling if my system is robust.
> 
> Best regards,
> Dennis
> 
> 
> On Mar 13, 2008, at 1:01 PM, Howard B wrote:
> 
> > Greetings all --
> >
> > Professional money managers are sometimes evaluated based on the  
> > Sharpe Ratio of their performance, so it has some value.  But, in 
my  
> > research, I have not found Sharpe Ratio to be a very good metric 
for  
> > use when developing systems.  Yes, higher Sharpe Ratios will 
have  
> > smaller standard deviations than lower Sharpe Ratios, but the  
> > standard deviation includes both positive and negative 
deviations.   
> > That is, it penalizes both positive and negative performance.  
If  
> > you are designing trend following systems with long holding 
periods,  
> > and looking for the infrequent large gains associated with this 
type  
> > of system, Sharpe Ratio penalizes these.  When Sharpe Ratio is 
used  
> > as the objective function in an automated walk forward process,  
> > systems selected as the best in-sample often perform much less 
well  
> > out-of-sample than systems selected using K-Ratio, RRR, CAR/MDD, 
or  
> > UPI.
> >
> > Thanks for listening,
> > Howard
> >
> >
> > On Wed, Mar 12, 2008 at 10:33 PM, Paul Ho <paultsho@xxx>  
> > wrote:
> >
> > Time doesnt permit me to write a long post. But I think Jack  
> > Schwager in one of his books povides a very good description of 
what  
> > You want. Tuschar Chande also has insights.
> > One such parameter is the Sharpe ratio, but you need use it 
slightly  
> > differently. Firstly, take risk free return as zero, and you are  
> > obtaining the ratio of mean return to std deviation. Secondly,  
> > calculated yearly sharpe ratios and compare them from year to 
year.
> >
> > From: amibroker@xxxxxxxxxxxxxxx 
[mailto:amibroker@xxxxxxxxxxxxxxx]  
> > On Behalf OfDennis Brown
> > Sent: Thursday, 13 March 2008 12:24 PM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: Re: [amibroker] Re: What is best statistic for 
straightness  
> > of equity curve?
> >
> > Brian,
> >
> > Thanks for your reply.
> >
> > My thinking is that the Std Error will work. I do not need to use 
a
> > Log function on my equity curve, because I do not compound my 
results,
> > so they are linear. I also base my work on constant range bars, so
> > that linearizes the curves even more. Profit potential can only 
come
> > from price movement. The smoothest and straightest equity curves 
come
> > from the most robust systems. Period. You can look at the curve 
and
> > judge it, or find a number that is associated with this property.
> >
> > However, step functions get introduced into your nice trading 
system
> > from big news events that change the character of the markets
> > overnight, or in a minute during the day. I consider these things
> > that produce large quick drawdowns will be captured by a Maximum
> > Drawdown metric. The test period needs to have some of these big
> > events in it. The event may be too quick to affect a large
> > statistical function much, giving a false sense of goodness to the
> > system. Or the perturbation might show up in a way that takes a 
great
> > system and makes the smoothness number look bad due to a one time
> > event. That is the challenge with a single number, so I will have 
to
> > experiment with the right weightings.
> >
> > That is why I say that the absolute judgement comes from 
examination
> > of the equity curve. The goodness numbers are just for ease of
> > relative comparisons of automated parameter optimization for 
candidate
> > systems. It is also nice to have a number or two as a future 
point of
> > reference rather than going back over equity curves for every
> > comparison.
> >
> > Perhaps an FFT over the equity curve would generate an interesting
> > signature in the period of the dominant frequency and I also need 
the
> > amplitude. I would have to look into this more, since I have not
> > tried this before.
> >
> > I will start out simple and see how better numbers compare to the
> > curves, then decide where to go from there.
> >
> > > (Why don't you just start posting some of your bits and pieces, 
like
> > > your new PlotShapes PDF, to the UKB - it is a live site - we 
don't
> > > have to wait for the big bang moment to become an author - a 
lot of
> > > my stuff is mundane and/or half finished, but it still has its  
> > uses).
> >
> > I am buried in work right now, so I wanted to gauge the value to
> > others of some of the things I could post on the UKB. I would 
have to
> > fight for the time to figure out how to post and fiddle with with
> > formatting issues etc. If it were as easy as sending a PDF email
> > attachment here, I would have done it a month ago. It is the up 
front
> > time investment that is holding me back right now.
> >
> > When I get little feedback or interest from a post, I can't 
prioritize
> > the time to share more of what I am doing. If I were not so busy, 
I
> > would do it anyway, but for now I need powerful justification to 
delay
> > some other important work to make time for it. This is not a spare
> > time hobby for me, because I have no spare time right now. :-(
> >
> > I could use a teammate to get me through the initial stages. 
However,
> > I see that only a few have ventured as far as posting yet, so the
> > field is limited. I do all my content creation on a Mac, and keep 
my
> > virtual PC free of everything but AmiBroker and related support
> > programs. That is why I prefer to generate PDF content as it works
> > everywhere. And I have exceptionally easy to use and powerful 
tools
> > for generating them already.
> >
> > Best regards,
> > Dennis Brown
> >
> > On Mar 12, 2008, at 7:19 PM, brian_z111 wrote:
> >
> > > Dennis,
> > >
> > > So where is your thinking on this now?
> > >
> > >
> > > (I have been following and I am building to some possible input 
but
> > > since I don't understand logs and barely understand standard 
error I
> > > have had to go back to school - it takes quite a while for me 
to get
> > > my head around that stuff and interpret it into trade talk).
> > >
> > > I have taken a different approach to evaluation (which is still 
a
> > > work in progress) and based on that I am inclined to the view 
that
> > > evaluations on one equity curve are on rather weak ground - IMO
> > > simulation is required for analysis of 'what counts most'.
> > >
> > > Also I am zeroing in on the root causes of equity curve 
profiles and
> > > measuring smoothness of a curve is measuring the effect.
> > >
> > > BTW - your pane based analysis is very interesting but I think
> > > ultimately it might prove to have some limitations for good
> > > evaluation (but not if we correctly identify root causes - we 
can
> > > just pick them out, add some mathematical antecedents and then 
we
> > > will now the answers that simulation will give us and not need 
to
> > > bother the processor - I have convinced myself that this is in 
my
> > > grasps and later I hope the maths people will connect my 
conceptual
> > > does and bingo, we are there).
> > >
> > > However, I love your question and approach, so over to your  
> > immediate
> > > problem (I had it in mind to go to town on an equity curve  
> > smoothness
> > > metric anyway).
> > >
> > > K-ratio is actually a risk reward metric (is that what you 
want)?
> > >
> > > It also (to me) gets a little mysterious in its workings 
(Klestner
> > > doesn't fully explain one part of it - not from my, lay, point 
of
> > > view anyway).
> > >
> > > I am still thinking about it.
> > >
> > > So far I would say StDev is out.
> > > StandardError will do exactly what you say you want to do (as 
far as
> > > I can tell - once again the stats teachers seem to find it hard 
to
> > > put it into trade talk - I see it explained in different ways in
> > > different books).
> > >
> > > I haven't reached a final conclusion but it seems most likely 
that  
> > if
> > > you use Standard Error on a compounded equity curve with the 
LogN
> > > approach taken by Klestner you are there - no need to go past 
that -
> > > my reservation is based on the fact that I am not sure how to 
handle
> > > standardisation - I only work in relative % change - Klestner
> > > attempts to standardise the K-ratio - he had some trouble with 
it to
> > > start out and had to add a standardising factor.
> > >
> > >> Everything I do is in indicator mode in realtime. I build all 
my
> > >> metrics into my AFL. My charts and numbers always match and all
> > >> my
> > >> settings are stored in my Flexible Parameters scheme for 
different
> > >> test systems. It is a little different approach, but that is 
one
> > >> of
> > >> the beauties of AB --that it allows a lot of flexibility of 
doing
> > >> your
> > >> own thing if you don't want to use the built-in ways.
> > >
> > > Yes, all of my evaluation methods are home made, or adaptions of
> > > popular methods - works for me.
> > >
> > > As I said - if you want all of your evaluation in one window you
> > > might need a math formula to sum up the transition from root 
cause  
> > to
> > > simulation (I naively believe I have the beginning and end in 
the  
> > bag
> > > and conceptually the middle formula seems attainable).
> > >
> > > (Why don't you just start posting some of your bits and pieces, 
like
> > > your new PlotShapes PDF, to the UKB - it is a live site - we 
don't
> > > have to wait for the big bang moment to become an author - a 
lot of
> > > my stuff is mundane and/or half finished, but it still has its  
> > uses).
> > >
> > > brian_z
> > >
> > >
> > > --- In amibroker@xxxxxxxxxxxxxxx, Dennis Brown <see3d@> wrote:
> > >>
> > >> Howard,
> > >>
> > >> Thanks for the input. I will investigate these some more.
> > >>
> > >> However, I do not use the built-in equity functions, or any of 
the
> > >> built-in trading functions. Tomasz has done a wonderful job 
with
> > >> these, but they do not fit well with what I am doing with my
> > > trading.
> > >> I find it easier to understand what I am getting if I write
> > > everything
> > >> myself just for my situation and not the general case.
> > >>
> > >> Everything I do is in indicator mode in realtime. I build all 
my
> > >> metrics into my AFL. My charts and numbers always match and all
> > > my
> > >> settings are stored in my Flexible Parameters scheme for 
different
> > >> test systems. It is a little different approach, but that is 
one
> > > of
> > >> the beauties of AB --that it allows a lot of flexibility of 
doing
> > > your
> > >> own thing if you don't want to use the built-in ways.
> > >>
> > >> Sometimes, you have to march to the beat of a different 
drummer to
> > >> make money in these markets.
> > >>
> > >> Thanks again,
> > >> Dennis Brown
> > >>
> > >>
> > >> On Mar 12, 2008, at 1:38 PM, Howard B wrote:
> > >>
> > >>> Hi Dennis --
> > >>>
> > >>> There are several metrics already built in to AmiBroker that
> > > measure
> > >>> both the steepness and smoothness of the equity curve. Try
> > >>> generating a few test runs, plot their equity curves, note the
> > >>> values of these metrics, and see which ones best fit your
> > > trading
> > >>> personality. A nice advantage to using these is that they
> > > usually
> > >>> tend to select trading systems that test well out-of-sample, 
so
> > > are
> > >>> appropriate for use with the Walk-Forward technique now also
> > > built
> > >>> in to AmiBroker.
> > >>>
> > >>> KRatio
> > >>> CAR/MDD
> > >>> RAR/MDD
> > >>> RRR
> > >>> RecoveryFactor
> > >>> UlcerPerformanceIndex
> > >>>
> > >>> Thanks,
> > >>> Howard
> > >>>
> > >>> On Tue, Mar 11, 2008 at 6:06 PM, Dennis Brown <see3d@>
> > >>> wrote:
> > >>> Hello,
> > >>>
> > >>> I have my system for intraday trading complete enough that I 
need
> > > to
> > >>> start selecting goodness criteria for comparing variations. I 
have
> > >>> selected a number of metrics to display in realtime for an n 
day
> > >>> backtest like:
> > >>>
> > >>> total trade count
> > >>> average bars per trade
> > >>> winning trade %
> > >>> trade bars % in green
> > >>> best trade $
> > >>> worst trade $
> > >>> average win $
> > >>> average loss $
> > >>> *total profit $
> > >>> *max draw down $
> > >>> *EDGE (average $ per trade)
> > >>> *I have a graph of the cumulative profit over time and an 
overlaid
> > >>> straight line plot. This is the most powerful tool, because it
> > > lets
> > >>> me see the real character of the system. The straighter the 
line,
> > > the
> > >>> less likely it is over fit to the data and represents a robust
> > > system.
> > >>>
> > >>> I also have a graph of the trade equity on a trade by trade
> > > basis, so
> > >>> I can see how good the entry timing is and how a trade 
progresses
> > > on
> > >>> average or in outlier conditions.
> > >>>
> > >>> The * items are my key metrics for system comparison. This 
simple
> > >>> system runs completely in indicator mode. I test about 1000-
2000
> > >>> trades over a 10 week test period.
> > >>>
> > >>> Because of the type and manner of my trades (1 futures 
contract
> > > only
> > >>> traded during market hours), the data is easy to judge for
> > > goodness.
> > >>> Since every day is an island, I could even use interesting 
random
> > > day
> > >>> strategies for in and out of sample data, but so far I just 
use
> > >>> various sequential segments.
> > >>>
> > >>> However, when I am spinning my scroll wheel on parameters 
while
> > >>> looking at my charts, it would be nice to have a number that
> > >>> represents how straight the equity curve is as a first pass --
> > >>> especially for when I partially automate the optimization
> > > process
> > >>> later.
> > >>>
> > >>> I thought I would just take the standard deviation of the 
whole
> > > curve
> > >>> to the straight line. This is easy. But I think some of you 
have
> > >>> given this problem a lot of thought and I figured one of you 
may
> > > have
> > >>> some additional insights into the best method for getting a
> > > meaningful
> > >>> number for straightness/smoothness of the equity curve. So 
here I
> > > put
> > >>> the question to you now with an open mind, before I become 
set in
> > > my
> > >>> ways ;-)
> > >>>
> > >>> Best regards,
> > >>> Dennis Brown
> > >>>
> > >>>
> > >>>
> > >>>
> > >>
> > >
> > >
> > >
> > >
> > > Please note that this group is for discussion between users 
only.
> > >
> > > To get support from AmiBroker please send an e-mail directly to
> > > SUPPORT {at} amibroker.com
> > >
> > > For NEW RELEASE ANNOUNCEMENTS and other news always check 
DEVLOG:
> > > http://www.amibroker.com/devlog/
> > >
> > > For other support material please check also:
> > > http://www.amibroker.com/support.html
> > >
> > > Yahoo! Groups Links
> > >
> > >
> > >
> >
> >
> >
> >
>



------------------------------------

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/