|
PureBytes Links
Trading Reference Links
|
Nick,
using semideviation seems to be a self-evident approach. Any idea how to code
that in AB?
Thanks
Thomas
On 08.10.2009, 18:01:36 NickW wrote:
> Hi,
>
> This is a great thread. Thanks for all the valuable information.
>
> Using SQN is something I've struggled with aswell where is punishes the
> outliers. When designing trend following systems, 5% of the trades are the
> big winners that makes trend following work, so you really don't want to
> punish the system for having few great winners. I am going to look more
> into semideviation to see how that can solve that problem.
>
> Thanks
> Nick
>
> On Thu, Oct 8, 2009 at 7:50 AM, Howard B <howardbandy@xxxxxxxxx> wrote:
> > Hi Mike --
> >
> > CAR is compound annual rate of return. Expectancy is the percentage (or
> > dollar amount, but not for this discussion) gain on the average trade.
> > The final value of the trading account, Terminal Relative Wealth in some
> > descriptions, is: (1 + expectancy) raised to the power of the number of
> > trades. When there are a few large trades included, this relationship is
> > slightly different, but not enough to worry this discussion. CAR is: (1
> > + annual gain) raised to the power of the number of years. These are
> > both based on geometric means. Aren't they the same, or so close that
> > they act the same when used as an objective function?
> >
> > Van Tharp defines System Quality Number on page 28 of "Definitive Guide
> > to Position Sizing." SQN = (expectancy / standard deviation) times
> > squareroot of number of trades. SQN decreases when a median trade is
> > replaced by either a large win or a large loss. It is possible to
> > redefine the metric so that large wins are not penalized, such as by
> > using the semi-deviation instead of the standard deviation as the
> > denominator. But without making that change, outliers, both good and
> > bad, do affect SQN by reducing it.
> >
> > What happens with open (or potentially open) trades at the boundaries of
> > walk forward periods is difficult to handle in both theory and practice.
> > Tomasz' implementation is to close all open trades at the end of each WF
> > period; and go into each WF period flat, not taking a new position until
> > there is a new signal. This creates a potentially serious distortion of
> > results when trades are typically held a long time (a large proportion of
> > a WF period) or when there are a few large trades that comprise the
> > majority of a system's profit or loss.
> >
> > I think this leads to a conclusion that we both agree on -- high quality
> > trading systems that can benefit from position sizing are based on high
> > frequency trading with very careful control over losses, and even control
> > over gains.
> >
> > Thanks for listening,
> > Howard
> >
> > On Wed, Oct 7, 2009 at 2:36 PM, Mike <sfclimbers@xxxxxxxxx> wrote:
> >> Howard,
> >>
> >> Assuming that SQN is the t-test for expectancy, then optimizing on the
> >> t-test of expectancy (i.e. SQN) is not the same as optimizing on CAR.
> >>
> >> The primary reason that CAR is a poor target for optimization is that
> >> outliers can significantly improve the calculation. The exact opposite
> >> is true for SQN.
> >>
> >> SQN rewards consistency and punishes outliers. Consistent winners with a
> >> few large wins will improve CAR but hurt SQN, resulting in different
> >> parameter combinations being selected during an optimization.
> >>
> >> As for writing a custom method, AmiBroker's stats are calculated based
> >> on the assumption that all open trades are closed out at the backtest
> >> boundary date. Many open trades, or even just a few large open trades,
> >> can skew these values.
> >>
> >> For high frequency strategies or strategies using heavy position sizing,
> >> creating a custom function is the only way to get reliable measurements.
> >>
> >> Mike
> >>
> >>
> >> --- In amibroker@xxxxxxxxxxxxxxx <amibroker%40yahoogroups.com>, Howard B
> >>
> >> <howardbandy@xxx> wrote:
> >> > Greetings all --
> >> >
> >> > There has been a lot of activity on this thread. I'll not respond to
> >>
> >> each
> >>
> >> > point individually, but will make a couple of general comments.
> >> >
> >> > I know David Aronson, speak with him regularly, and collaborate with
> >> > him
> >>
> >> on
> >>
> >> > projects. I have a copy of his book, "Evidence-Based Technical
> >>
> >> Analysis."
> >>
> >> > His book is excellent and I highly recommend it. I think David and I
> >> > are
> >>
> >> in
> >>
> >> > pretty close agreement on most of the modeling, simulation, testing,
> >> > and validation issues.
> >> >
> >> > I have spoken with Robert Pardo and have exchanged several emails and
> >>
> >> forum
> >>
> >> > postings with him. I think his earlier book was very good,
> >> > particularly
> >>
> >> at
> >>
> >> > the time it was published. And his more recent book is not quite up to
> >> > those standards. There are several important areas he did not cover
> >> > and several areas where I see things considerably differently than
> >> > Robert.
> >> >
> >> > I have spoken with and exchanged emails with Van Tharp, and I have
> >>
> >> copies of
> >>
> >> > his books "Trade Your Way to Financial Freedom" and "Definitive Guide
> >> > to Position Sizing." Both are excellent, and I recommend them both
> >> > highly.
> >>
> >> Be
> >>
> >> > sure to get the second edition of Trade Your Way to Financial Freedom
> >> > --
> >>
> >> it
> >>
> >> > has some important corrections and clarifications.
> >> >
> >> > Permit me a short rant on my soapbox. I really dislike it when people
> >>
> >> claim
> >>
> >> > ownership of common terms. Tom DeMark, Robert Pardo, Van Tharp, and
> >>
> >> others
> >>
> >> > put Service Mark symbols on terms that they think are unique to them,
> >>
> >> but
> >>
> >> > are not. I appreciate Tharp's enthusiasm over what he calls System
> >>
> >> Quality
> >>
> >> > Number, but I wish he would not put the Service Mark symbol next to
> >>
> >> every
> >>
> >> > occurrence of it. And trying to Service Mark the term Position Sizing
> >> > is like a dietician service marking "calorie counting." Robert Pardo
> >> > claims "Walk Forward." I used exactly that term describing exactly
> >> > that process
> >>
> >> in
> >>
> >> > research papers I delivered at conferences in the late 1960s. The mark
> >>
> >> has
> >>
> >> > been registered, not by Robert, but by a company I used to work for
> >> > and
> >>
> >> with
> >>
> >> > which Robert was not associated, over my strong objection. End of
> >> > rant.
> >> >
> >> > System quality number is equivalent to t-test. Systems with SQNs above
> >> > 2 work well for exactly the same reasons that systems with t-test
> >> > scores
> >>
> >> above
> >>
> >> > 2 work well. In fact, it is possible to create a custom objective
> >>
> >> function
> >>
> >> > that Is the t-test and use it for optimization. Attendees at my
> >>
> >> workshops
> >>
> >> > in Melbourne later this month will see that demonstrated. Optimizing
> >> > for the t-test of expectancy is equivalent to optimizing for CAR, so
> >> > don't bother creating the custom function unless you have a better
> >> > candidate
> >>
> >> for
> >>
> >> > your objective function than CAR.
> >> >
> >> > Back to the topic at hand -----
> >> >
> >> > There is No rule of thumb to determine how long the in-sample period
> >>
> >> should
> >>
> >> > be. The Only way to determine that is by testing the model and the
> >> > data together. And be prepared for that length to change over time.
> >> > Some writers suggest a relationship between the number of free
> >> > parameters and
> >>
> >> the
> >>
> >> > number of data points, or some proportional division of the available
> >>
> >> data.
> >>
> >> > Those techniques do work on industrial time-series data which is
> >> > usually stationary, but they do not work on financial time-series data
> >> > which is non-stationary and changes as trading systems become better
> >> > at
> >>
> >> extracting
> >>
> >> > inefficiencies from it.
> >> >
> >> > No matter how good the in-sample results look, no matter how high the
> >>
> >> t-test
> >>
> >> > score is, no matter how many closed trades are represented --
> >> > in-sample results have no value in estimating the future performance
> >> > of the
> >>
> >> system.
> >>
> >> > None. The only information you have that gives any indication of
> >> > future performance are the out-of-sample results from testing on data
> >> > that was never used at all -- not even once -- during system
> >> > development.
> >> >
> >> > Tomorrow is out-of-sample. The only way to prepare for real-money
> >>
> >> trading
> >>
> >> > tomorrow is to be rigorous during the system testing and validation
> >> > process. Anything less will overestimate the probability of success.
> >> >
> >> > Thanks for listening,
> >> > Howard
>
------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|