|
PureBytes Links
Trading Reference Links
|
Mike,
thanks for your answer!
On 06.10.2009, 08:47:28 Mike wrote:
> Thomas,
>
> When I was reading "Van Tharp's Definitive Guide To Position Sizing", I put
> togeather a web page hosting a position sizing simulator for the examples
> in the book. I wrote the simulator for usage against my own strategy. Then
> hosted it for anyone else to compare against the book.
>
> You can play with it to get a feel for how different SQN values affect
> position sizing:
>
> http://www.afltools.com/positionsize.html
>
> If you do play with the simulator, and have any questions, please send them
> to the email address provided on that page, as opposed to on this forum.
Very interesting - I will have a close look inti it!
>
> To answer your question;
>
> SQN = (Expectancy / Standard deviation of R) x max(10, square root of
> number of trades)
Since I don't have the book I found a different formula on
http://www.ninjatrader-support2.com/vb/showthread.php?t=4320 which may not be
correct.
>
> The most precise calculation of Expectancy (according to Tharp) is the summ
> of all closed trade R multiples.
>
> Ideally you would have a precise R for each trade from which to calculate
> the closing R multiple. Otherwise, you can use the average loss as R (e.g.
> like when using a moving average crossover for exit there is no
> predetermined R like there would be when using a stoploss).
>
> If you are unable to sum the R multiples, you can use
>
> Expectancy = ((Avg. Profit x Prob. of win) - (Avg. Loss x Prob. of loss)) /
> Avg. Risk
There might be different definitions of expectancy. According to Howard's book
QTS (p. 319) Expectancy is equivalent with Average Profit/Loss already reported
in Amibroker.
>
> Unfortunately, for this usage, the summary statistics produced by AmiBroker
> are based on the assumption that open trades are closed out at the current
> price. This skews the values. So, you must write custom backtest code to
> only work with *closed* trades to figure avg. win, avg. loss, etc.
>
> Also, your usage of StDev is not correct. You must calculate the R
> multiple for each closed trade, then calculate the standard deviation of
> those values yourself.
I will investigate how to do this. Example 3 on
http://www.amibroker.net/guide/a_custommetrics.html should be a good place to
start.
>
> Hope that helps
>
> Mike
Thanks again.
Thomas
>
> --- In amibroker@xxxxxxxxxxxxxxx, Thomas Ludwig <Thomas.Ludwig@xxx> wrote:
> > Mike,
> >
> > how do calculate SQN? I've tried this:
> >
> > SQN=sqrt(st.GetValue("AllQty"))*st.GetValue("AllAvgProfitLoss")/StDev(st.
> >GetValue("NetProfit")); bo.AddCustomMetric("SQN",SQN);
> >
> > but I'm not sure if this is correct.
> >
> > What do you think?
> >
> > Regards,
> >
> > Thomas
> >
> > On 05.10.2009, 09:29:41 Mike wrote:
> > > Ton,
> > >
> > > Are you saying that you have not found an IS/OOS pair that works well?
> > > What measure are you using to judge "stability" of the walk forward
> > > process (i.e. what measure are you using to judge the process as
> > > random)?
> > >
> > > After testing with multiple IS periods, and with multiple OOS periods,
> > > I was able to identify "fixed" window lengths that proved more
> > > consistent than the others tested.
> > >
> > > I reached this conclusion by charting a distribution curve of CAR for
> > > the OOS results. My fitness function is currently based on UPI, and
> > > thus my walk forward is driven by that value. However, ultimately my
> > > interest is in how consistent CAR would be which is why I used that for
> > > evaluating the goodness of fit for the IS/OOS period lengths.
> > >
> > > In my case, over a 13 year period, a 2 year IS and 6 month OOS (for a
> > > total of 26 OOS data points) produced the most normal looking
> > > distribution of CAR results (i.e. central peak, smallest standard
> > > deviation). Excluding the results from all of 1999 and the first half
> > > of 2000 (during which results were abnormally strong), the distribution
> > > curve looks even better.
> > >
> > > Also, have you tried working with different fitness functions? Perhaps
> > > your fitness function doesn't adequately identify the "signal" and thus
> > > misguides the walk forward, regardless of IS/OOS window lengths.
> > >
> > > I am in the process of running a new walk forward over the last 7.5
> > > years using Van Tharp's System Quality Number (SQN) as my fitness
> > > function. I have kept the same 2 year IS/6 months OOS for a total of 15
> > > OOS data points. My system strives to generate a minimum average of 2
> > > trades per day, so each IS period generally has 1000 or more trades
> > > from which to calculate the fitness.
> > >
> > > It has not run to completion yet. But, for the periods that have
> > > produced results, the results look promising (at least with respect to
> > > the SQN of the OOS relative to the SQN of the IS, I have not yet
> > > created the distribution of CAR for OOS).
> > >
> > > Assuming that the remainder of the results are equally strong, I will
> > > walk forward further back in history to get the full 26 data points to
> > > compare against the results produced using my UPI fitness. If the CAR
> > > distribution is more normal using SQN as fitness, then I will
> > > officially start using SQN for generating optimal values for my next
> > > live OOS.
> > >
> > > If you are willing to share, I would be curious to hear if SQN as a
> > > fitness function was able to produce a more stable walk forward for
> > > you, and what measure you are using to judge "stable".
> > >
> > > Mike
> > >
> > > --- In amibroker@xxxxxxxxxxxxxxx, "Ton Sieverding" <ton.sieverding@>
> >
> > wrote:
> > > > Hi Howard,
> > > >
> > > > I still am struggling with the following sentence from David
> > > > Aronson
> > > >
> > > > : "The decision about how to apportion the data between the IS and
> > > > : OOS
> > > >
> > > > subsets is arbitrary. There is no theory that suggests what fraction
> > > > of the data should be assigned to training ( IS ) and testing ( OOS
> > > > ). Results can be very sensitive to these choices ... ". Because this
> > > > is exactly what I am seeing. WalkFoward results are more then
> > > > sensitive to the IS/OOS relation and in many cases a pure random
> > > > story. I am getting more and more the feeling that WalkForward is not
> > > > the correct or better objective way to test trading systems. With all
> > > > respect to Robert Pardo's idea's about this topic and what you are
> > > > writing in QTS ...
> > > >
> > > > Regards, Ton.
> > > >
> > > >
> > > > ----- Original Message -----
> > > > From: Howard B
> > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > Sent: Monday, October 05, 2009 12:48 AM
> > > > Subject: Re: [amibroker] Re: Is the Walk forward study useful?
> > > >
> > > >
> > > > Greetings all --
> > > >
> > > > My point of view on the length of the in-sample and out-of-sample
> > > > may be a little different.
> > > >
> > > > The logic of the code has been designed to recognize some pattern
> > > > or characteristic of the data. The length of the in-sample period is
> > > > however long it takes to keep the model (the logic) in
> > > > synchronization with the data. There is no one answer to what that
> > > > length is. When the pattern changes, the model fits it less well.
> > > > When the pattern changes significantly, the model must be
> > > > re-synchronized. The only person who can say whether the length is
> > > > correct or should be longer or shorter is the person running the
> > > > tests.
> > > >
> > > > The length of the out-of-sample period is however long the model
> > > > and the data remain in sync. That must be some length of time beyond
> > > > the in-sample period in order to make profitable trades. It could
> > > > be a long time, in which case there is no need to modify the model at
> > > > all during that period. There is no general relationship between the
> > > > length of the in-sample period and the length of the out-of-sample
> > > > period -- none. There is no general relationship between the
> > > > performance in-sample and the performance out-of-sample. The greater
> > > > the difference between the two, the better the system has been fit to
> > > > the data over the in-sample period. But that does not necessarily
> > > > mean that the out-of-sample results are less meaningful.
> > > >
> > > > You can perform some experiments to see what the best in-sample
> > > > length is. And then to see what the typical out-of-sample length is.
> > > > Knowing these two, set up a walk forward run using those lengths.
> > > > After the run is over, ignore the in-sample results. They have no
> > > > value in estimating the future performance of the system. It is the
> > > > out-of-sample results that can give you some idea of how the system
> > > > might act when traded with real money.
> > > >
> > > > It is nice to have a lot of closed traded in the out-of-sample
> > > > period, but you can run statistics on as few as 5 or 6. Having fewer
> > > > trades means that it will be more difficult to achieve statistical
> > > > significance. The number 30 is not magic -- it is just conventional.
> > > >
> > > > I think it helps to distinguish between the in-sample and
> > > > out-of-sample periods this way -- in-sample is seeing how well the
> > > > model can be made to fit the older data, out-of-sample is seeing how
> > > > well it might fit future data.
> > > >
> > > > Ignore the television ads where person after person exclaims
> > > > "backtesting!" as though that is the key to system development. It
> > > > is not. Backtesting by itself, without going on to walk forward
> > > > testing, will give the trading system developer the impression that
> > > > the system is good. In-sample results are always good. We do not
> > > > stop fooling with the system until they are good. But in-sample
> > > > results have no value in predicting future performance -- none.
> > > >
> > > > There are some general characteristics of trading systems that make
> > > > them easier to validate. Those begin with having a positive
> > > > expectancy -- no system can be profitable in the long term unless it
> > > > has a positive expectancy. Then going on to include trade
> > > > frequently, hold a short time, minimize losses. Of course, there
> > > > have been profitable systems that trade infrequently, hold a long
> > > > time, and suffer deep drawdowns. It is much harder to show that
> > > > those were profitable because they were good rather than lucky.
> > > >
> > > > There is more information about in-sample, out-of-sample, walk
> > > > forward testing, statistical validation, objective functions, and so
> > > > forth in my book, "Quantitative Trading Systems."
> > > > http://www.quantitativetradingsystems.com/
> > > >
> > > > Thanks for listening,
> > > > Howard
> > > >
> > > >
> > > >
> > > > On Sun, Oct 4, 2009 at 10:56 AM, Bisto <bistoman73@> wrote:
> > > >
> > > >
> > > > Yes, I believe that you should increase the IS period
> > > >
> > > > as general rule is not true "the shortest the best" trying to
> > > > catch every market change because it's possible that a too short IS
> > > > period produces a too low number of trades with no statistical
> > > > robustness --> you will find parameters that are more likely
> > > > candidated to fail in OS
> > > >
> > > > try a longer IS period and let's see what will happen
> > > >
> > > > I read an interesting book on this issue: "The evaluation and
> > > > optimization of trading strategies" by Pardo. Maybe he repeated too
> > > > much times the same concepts nevertheless I liked it
> > > >
> > > > if anyone could suggest a better book about this issue it would
> > > > be very appreciated
> > > >
> > > >
> > > >
> > > > Bisto
> > > >
> > > > --- In amibroker@xxxxxxxxxxxxxxx, "Gonzaga" <gonzagags@> wrote:
> > > > > Oh, sorry, I am lost in translation ... ;-)
> > > > > Yes I meant trades of my IS period.
> > > > > I've got about 70 trades in my IS period, three months.
> > > > > BUT, I buy stocks in a multiposition way.This means, that my
> > > > > hole capital divides among several stocks purchased
> > > > > simultaneously. So, in my statistics, I use to average my
> > > > > trades. When I use maxopenpositions=7, I use to average my
> > > > > results every 7 trades. Considering that, my trades in three
> > > > > months are not 70, but less ( not exactly 70/7, but less than
> > > > > 70)
> > > > >
> > > > > If I use maxopenposition=1, which is, invest all my capital
> > > > > every trade, in three months I would have about 29 trades. So I
> > > > > suppose I have to increase the IS period.. isn`t it?
> > > > >
> > > > > --- In amibroker@xxxxxxxxxxxxxxx, "Bisto" <bistoman73@> wrote:
> > > > > > What do you mean with "I don't have many buyings and
> > > > > > sellings"?
> > > > > >
> > > > > > If you have less than 30 trades in an IS period, IMHO, you
> > > > > > are using a too short period due to not statistical
> > > > > > robustness --> WFA is misleading, try a longer IS period
> > > > > >
> > > > > > Bisto
> > > > > >
> > > > > > --- In amibroker@xxxxxxxxxxxxxxx, "Gonzaga" <gonzagags@>
wrote:
> > > > > > > Thanks for the answers
> > > > > > > To Keith McCombs :
> > > > > > >
> > > > > > > I use 3 months IS test and 1 month step, this is, 1 month
> > > > > > > OS test. My system is an end-of day-system, so I don't have
> > > > > > > many buyings and sellings.. Perhaps I should make bigger
> > > > > > > the IS period?
> > > > > > >
> > > > > > > anyway, my parameter behaves well in any period. Of course
> > > > > > > it is an optimized variable, but it doesn't fail in ten
> > > > > > > years, in none of those ten years, over 500 stocks.. a very
> > > > > > > long period.. So, couldn't it be better, on the long run,
> > > > > > > than the parameters optimized with the WF study? (In fact,
> > > > > > > I am using it now, the optimized variable)
> > > > > > > That's my real question..
> > > > > > >
> > > > > > > To dloyer123:
> > > > > > > I haven't understood the meaning of the Walk Forward
> > > > > > > Efficency, and seems interesting. can you explain it
> > > > > > > better, please..?
> > > > > > >
> > > > > > > --- In amibroker@xxxxxxxxxxxxxxx, "dloyer123" <dloyer123@>
> >
> > wrote:
> > > > > > > > I have had similar experiences. I like to use WFT to
> > > > > > > > estimate what Pardo call's his "Walk Forward Efficency",
> > > > > > > > or the ratio of the out of sample WF profits to just
> > > > > > > > optimizing over the entire time period.
> > > > > > > >
> > > > > > > > A good system should have as high a WFE as posible.
> > > > > > > > Systems with a poor WFE tend to do poorly in live
> > > > > > > > trading.
> > > > > > > >
> > > > > > > > If you have a parm set that works well over a long period
> > > > > > > > of live trading, then you are doing well!
> > >
> > > ------------------------------------
> > >
> > > **** IMPORTANT PLEASE READ ****
> > > This group is for the discussion between users only.
> > > This is *NOT* technical support channel.
> > >
> > > TO GET TECHNICAL SUPPORT send an e-mail directly to
> > > SUPPORT {at} amibroker.com
> > >
> > > TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
> > > http://www.amibroker.com/feedback/
> > > (submissions sent via other channels won't be considered)
> > >
> > > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > > http://www.amibroker.com/devlog/
> > >
> > > Yahoo! Groups Links
>
> ------------------------------------
>
> **** IMPORTANT PLEASE READ ****
> This group is for the discussion between users only.
> This is *NOT* technical support channel.
>
> TO GET TECHNICAL SUPPORT send an e-mail directly to
> SUPPORT {at} amibroker.com
>
> TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
> http://www.amibroker.com/feedback/
> (submissions sent via other channels won't be considered)
>
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> Yahoo! Groups Links
>
>
>
------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|