|
PureBytes Links
Trading Reference Links
|
Howard,
Assuming that SQN is the t-test for expectancy, then optimizing on the t-test of expectancy (i.e. SQN) is not the same as optimizing on CAR.
The primary reason that CAR is a poor target for optimization is that outliers can significantly improve the calculation. The exact opposite is true for SQN.
SQN rewards consistency and punishes outliers. Consistent winners with a few large wins will improve CAR but hurt SQN, resulting in different parameter combinations being selected during an optimization.
As for writing a custom method, AmiBroker's stats are calculated based on the assumption that all open trades are closed out at the backtest boundary date. Many open trades, or even just a few large open trades, can skew these values.
For high frequency strategies or strategies using heavy position sizing, creating a custom function is the only way to get reliable measurements.
Mike
--- In amibroker@xxxxxxxxxxxxxxx, Howard B <howardbandy@xxx> wrote:
>
> Greetings all --
>
> There has been a lot of activity on this thread. I'll not respond to each
> point individually, but will make a couple of general comments.
>
> I know David Aronson, speak with him regularly, and collaborate with him on
> projects. I have a copy of his book, "Evidence-Based Technical Analysis."
> His book is excellent and I highly recommend it. I think David and I are in
> pretty close agreement on most of the modeling, simulation, testing, and
> validation issues.
>
> I have spoken with Robert Pardo and have exchanged several emails and forum
> postings with him. I think his earlier book was very good, particularly at
> the time it was published. And his more recent book is not quite up to
> those standards. There are several important areas he did not cover and
> several areas where I see things considerably differently than Robert.
>
> I have spoken with and exchanged emails with Van Tharp, and I have copies of
> his books "Trade Your Way to Financial Freedom" and "Definitive Guide to
> Position Sizing." Both are excellent, and I recommend them both highly. Be
> sure to get the second edition of Trade Your Way to Financial Freedom -- it
> has some important corrections and clarifications.
>
> Permit me a short rant on my soapbox. I really dislike it when people claim
> ownership of common terms. Tom DeMark, Robert Pardo, Van Tharp, and others
> put Service Mark symbols on terms that they think are unique to them, but
> are not. I appreciate Tharp's enthusiasm over what he calls System Quality
> Number, but I wish he would not put the Service Mark symbol next to every
> occurrence of it. And trying to Service Mark the term Position Sizing is
> like a dietician service marking "calorie counting." Robert Pardo claims
> "Walk Forward." I used exactly that term describing exactly that process in
> research papers I delivered at conferences in the late 1960s. The mark has
> been registered, not by Robert, but by a company I used to work for and with
> which Robert was not associated, over my strong objection. End of rant.
>
> System quality number is equivalent to t-test. Systems with SQNs above 2
> work well for exactly the same reasons that systems with t-test scores above
> 2 work well. In fact, it is possible to create a custom objective function
> that Is the t-test and use it for optimization. Attendees at my workshops
> in Melbourne later this month will see that demonstrated. Optimizing for
> the t-test of expectancy is equivalent to optimizing for CAR, so don't
> bother creating the custom function unless you have a better candidate for
> your objective function than CAR.
>
> Back to the topic at hand -----
>
> There is No rule of thumb to determine how long the in-sample period should
> be. The Only way to determine that is by testing the model and the data
> together. And be prepared for that length to change over time. Some
> writers suggest a relationship between the number of free parameters and the
> number of data points, or some proportional division of the available data.
> Those techniques do work on industrial time-series data which is usually
> stationary, but they do not work on financial time-series data which is
> non-stationary and changes as trading systems become better at extracting
> inefficiencies from it.
>
> No matter how good the in-sample results look, no matter how high the t-test
> score is, no matter how many closed trades are represented -- in-sample
> results have no value in estimating the future performance of the system.
> None. The only information you have that gives any indication of future
> performance are the out-of-sample results from testing on data that was
> never used at all -- not even once -- during system development.
>
> Tomorrow is out-of-sample. The only way to prepare for real-money trading
> tomorrow is to be rigorous during the system testing and validation
> process. Anything less will overestimate the probability of success.
>
> Thanks for listening,
> Howard
>
------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|