|
PureBytes Links
Trading Reference Links
|
Hi Howard,
I'm a bit confused by your remarks.
1. You have said that Definitive Guide To Positoin Sizing is an excellent book.
2. You have said that SQN is a t-test (of expectancy?).
3. You have said that a t-test of expectancy is functionally equivalent to CAR.
4. You have said that CAR is a poor fitness function.
Given that the book repeatedly promotes usage of SQN as a comparator (i.e. a fitness function) with which to measure two systems, how do you reconcile point 1 with points 2-4?
Just to pin it down without any indirection, are you saying that you consider SQN as poor a fitness function as CAR? Or, have I misunderstood something you've said?
Thanks,
Mike
--- In amibroker@xxxxxxxxxxxxxxx, Howard B <howardbandy@xxx> wrote:
>
> Hi Mike --
>
> CAR is compound annual rate of return. Expectancy is the percentage (or
> dollar amount, but not for this discussion) gain on the average trade. The
> final value of the trading account, Terminal Relative Wealth in some
> descriptions, is: (1 + expectancy) raised to the power of the number of
> trades. When there are a few large trades included, this relationship is
> slightly different, but not enough to worry this discussion. CAR is: (1 +
> annual gain) raised to the power of the number of years. These are both
> based on geometric means. Aren't they the same, or so close that they act
> the same when used as an objective function?
>
> Van Tharp defines System Quality Number on page 28 of "Definitive Guide to
> Position Sizing." SQN = (expectancy / standard deviation) times squareroot
> of number of trades. SQN decreases when a median trade is replaced by
> either a large win or a large loss. It is possible to redefine the metric
> so that large wins are not penalized, such as by using the semi-deviation
> instead of the standard deviation as the denominator. But without making
> that change, outliers, both good and bad, do affect SQN by reducing it.
>
> What happens with open (or potentially open) trades at the boundaries of
> walk forward periods is difficult to handle in both theory and practice.
> Tomasz' implementation is to close all open trades at the end of each WF
> period; and go into each WF period flat, not taking a new position until
> there is a new signal. This creates a potentially serious distortion of
> results when trades are typically held a long time (a large proportion of a
> WF period) or when there are a few large trades that comprise the majority
> of a system's profit or loss.
>
> I think this leads to a conclusion that we both agree on -- high quality
> trading systems that can benefit from position sizing are based on high
> frequency trading with very careful control over losses, and even control
> over gains.
>
> Thanks for listening,
> Howard
>
>
> On Wed, Oct 7, 2009 at 2:36 PM, Mike <sfclimbers@xxx> wrote:
>
> >
> >
> > Howard,
> >
> > Assuming that SQN is the t-test for expectancy, then optimizing on the
> > t-test of expectancy (i.e. SQN) is not the same as optimizing on CAR.
> >
> > The primary reason that CAR is a poor target for optimization is that
> > outliers can significantly improve the calculation. The exact opposite is
> > true for SQN.
> >
> > SQN rewards consistency and punishes outliers. Consistent winners with a
> > few large wins will improve CAR but hurt SQN, resulting in different
> > parameter combinations being selected during an optimization.
> >
> > As for writing a custom method, AmiBroker's stats are calculated based on
> > the assumption that all open trades are closed out at the backtest boundary
> > date. Many open trades, or even just a few large open trades, can skew these
> > values.
> >
> > For high frequency strategies or strategies using heavy position sizing,
> > creating a custom function is the only way to get reliable measurements.
> >
> > Mike
> >
> >
> > --- In amibroker@xxxxxxxxxxxxxxx <amibroker%40yahoogroups.com>, Howard B
> > <howardbandy@> wrote:
> > >
> > > Greetings all --
> > >
> > > There has been a lot of activity on this thread. I'll not respond to each
> > > point individually, but will make a couple of general comments.
> > >
> > > I know David Aronson, speak with him regularly, and collaborate with him
> > on
> > > projects. I have a copy of his book, "Evidence-Based Technical Analysis."
> > > His book is excellent and I highly recommend it. I think David and I are
> > in
> > > pretty close agreement on most of the modeling, simulation, testing, and
> > > validation issues.
> > >
> > > I have spoken with Robert Pardo and have exchanged several emails and
> > forum
> > > postings with him. I think his earlier book was very good, particularly
> > at
> > > the time it was published. And his more recent book is not quite up to
> > > those standards. There are several important areas he did not cover and
> > > several areas where I see things considerably differently than Robert.
> > >
> > > I have spoken with and exchanged emails with Van Tharp, and I have copies
> > of
> > > his books "Trade Your Way to Financial Freedom" and "Definitive Guide to
> > > Position Sizing." Both are excellent, and I recommend them both highly.
> > Be
> > > sure to get the second edition of Trade Your Way to Financial Freedom --
> > it
> > > has some important corrections and clarifications.
> > >
> > > Permit me a short rant on my soapbox. I really dislike it when people
> > claim
> > > ownership of common terms. Tom DeMark, Robert Pardo, Van Tharp, and
> > others
> > > put Service Mark symbols on terms that they think are unique to them, but
> > > are not. I appreciate Tharp's enthusiasm over what he calls System
> > Quality
> > > Number, but I wish he would not put the Service Mark symbol next to every
> > > occurrence of it. And trying to Service Mark the term Position Sizing is
> > > like a dietician service marking "calorie counting." Robert Pardo claims
> > > "Walk Forward." I used exactly that term describing exactly that process
> > in
> > > research papers I delivered at conferences in the late 1960s. The mark
> > has
> > > been registered, not by Robert, but by a company I used to work for and
> > with
> > > which Robert was not associated, over my strong objection. End of rant.
> > >
> > > System quality number is equivalent to t-test. Systems with SQNs above 2
> > > work well for exactly the same reasons that systems with t-test scores
> > above
> > > 2 work well. In fact, it is possible to create a custom objective
> > function
> > > that Is the t-test and use it for optimization. Attendees at my workshops
> > > in Melbourne later this month will see that demonstrated. Optimizing for
> > > the t-test of expectancy is equivalent to optimizing for CAR, so don't
> > > bother creating the custom function unless you have a better candidate
> > for
> > > your objective function than CAR.
> > >
> > > Back to the topic at hand -----
> > >
> > > There is No rule of thumb to determine how long the in-sample period
> > should
> > > be. The Only way to determine that is by testing the model and the data
> > > together. And be prepared for that length to change over time. Some
> > > writers suggest a relationship between the number of free parameters and
> > the
> > > number of data points, or some proportional division of the available
> > data.
> > > Those techniques do work on industrial time-series data which is usually
> > > stationary, but they do not work on financial time-series data which is
> > > non-stationary and changes as trading systems become better at extracting
> > > inefficiencies from it.
> > >
> > > No matter how good the in-sample results look, no matter how high the
> > t-test
> > > score is, no matter how many closed trades are represented -- in-sample
> > > results have no value in estimating the future performance of the system.
> > > None. The only information you have that gives any indication of future
> > > performance are the out-of-sample results from testing on data that was
> > > never used at all -- not even once -- during system development.
> > >
> > > Tomorrow is out-of-sample. The only way to prepare for real-money trading
> > > tomorrow is to be rigorous during the system testing and validation
> > > process. Anything less will overestimate the probability of success.
> > >
> > > Thanks for listening,
> > > Howard
> > >
> >
> >
> >
>
------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|