[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Understanding Portfolio Backtest Reports ..was..Re: PositionScore Ideas



PureBytes Links

Trading Reference Links

Also,

Dangers Of The True Optimal f 

Do Equations Tell The Whole Story? 

by Gordon Gustafson 

----------------------------------------------------------------------
----------

Success in systems trading depends on having good trading systems and 
good money management practices. Optimal f can help you. 
Once you have a good trading system, wouldn't it be wonderful if you 
had an equation you could plug into a system to discover the perfect 
amount to risk on each trade? All you'd have to do would be to plug 
the results into a risk calculation and relax, knowing that the 
chances of ever losing again were practically nil. 

It's a nice idea. But the fact of the matter? Such equations could 
easily lead you astray. 

OPTIMAL F

When it comes to money management, one of the most popular topics is 
optimal f. Based on the historical results of a trading system, 
optimal f defines the optimal fixed fraction of total trading capital 
that should be allocated to any particular trade in order to maximize 
the geometric growth of the account. Dividing this by the largest 
loss results in the number of contracts that should be traded. 
According to this rationale, if you bet more you will go broke, and 
if you bet less, you will stay poor. 

Arguments made for the optimal f sound so positive - maximum 
geometric return! - you feel like a fool if you don't use it. Who 
doesn't want maximum account growth? Money management means devising 
a way to stay in the game. If there is a reasonable chance of going 
bust, then you need to adjust the amount of capital to trade lower. 
There will always be a chance of losing the account if you trade, but 
you should work to make sure the chances are small.

win%	loss%	Optimal f %
51	49	2
52	48	4
53	47	6
54	46	8
55	45	10
56	44	12
57	43	14
58	42	16
59	41	18
60	40	20
61	39	22
62	38	24
63	37	26
64	36	28
65	35	30
66	34	32
67	33	34
68	32	36
69	31	38
70	30	40
71	29	42
72	28	44
73	27	46
74	26	48
75	25	50
76	24	52
77	23	54
78	22	56
79	21	58
80	20	60
81	19	62
82	18	64
83	17	66
84	16	68
85	15	70
86	14	72
87	13	74
88	12	76
89	11	78
90	10	80
91	9	82
92	8	84
93	7	86
94	6	88
95	5	90
96	4	92
97	3	94
98	2	96
99	1	98
100	0	100


Figure 1: True optimal f. A really good system could actually wipe 
you out completely.

Also,

Buy-And-Hold Comparisons To Evaluate Stock Trading

----------------------------------------------------------------------
----------
by Jack Schwager 

----------------------------------------------------------------------
----------
Benchmark comparisons are a standard technique for the performance of 
a trading system. One particular benchmark is the buy-and-hold 
approach. This noted market analyst looks at the steps to using it. 
----------------------------------------------------------------------
----------
Just because a trading system makes money in the stock market doesn't 
mean it's a good system. After all, it is possible for a system to do 
well but still fall short of the results that could have been 
realized by a simple buy-and-hold approach. The key question in 
testing a stock market system is: How does the system compare with 
the buy-and-hold approach? 


FIGURE 1: DUPONT DE NEMOURS & CO. Each time the constant share size 
system flashes a buy or sell signal, the new trade has a unique 
number of shares for the trade entered. Therefore, the buy-and-hold 
system must have the number of shares held adjusted to match the 
number of shares held by the the constant share system.
THE PROBLEM 
Answering this question is not as simple as it might appear. In my 
previous article, we showed why a constant share size trade 
assumption led to severe distortions in testing trading systems and 
why a constant dollar trade size assumption was far preferable. The 
problem, however, is that system results based on a constant dollar 
trade size -- for example, $1,000 -- cannot be directly compared with 
buy-and-hold results for the same number of shares as the first 
trade. 

To understand why, assume a stock is trading at $5 at the start of 
the test period before it then advances to $50 by the end of the 
survey period. In this case, 200 shares, which equal $1,000 at the 
start, would equal $10,000 at the end. In contrast, the constant 
dollar trade size system results would continue to assume a $1,000 
trade size on each signal. 

As a result, the buy-and-hold case would assume a much larger average 
position size as time elapsed. Consequently, a comparison between the 
system results and buy-and-hold results would be that of the 
proverbial apples and oranges. 

A TWO-STEP SOLUTION 

Two possible adjustment methods can be used to allow valid 
comparisons between trading system results and buy-and-hold results. 

Method 1 „ Adjust the constant share system trade size to reflect 
equity growth. This can be done by compounding the number of shares 
per trade using a factor based on the current equity. Multiply the 
ratio of $1,000/price by the ratio of current equity to the assumed 
starting equity. If the starting equity equals $1,000, then this 
factor ratio is equal to ($1,000 + net profit)/$1,000. Therefore, the 
number of shares is given by this formula: 

Number of shares = ($1,000/price)($1,000 + Net profit)/$1,000 

Thus, if the system made $2,000, the second factor in the equation 
would equal 3, because there would be $3,000 available instead of 
$1,000. This approach breaks down if the loss exceeds $1,000, since 
it would result in a negative equity ratio, which in turn would imply 
a negative number of shares -- a nonsensical result. 

Next, we will need a percent return comparison with the buy-and-hold 
case, which should use a compounded percent return statistic. This is 
given by the following formula: 

Compounded percent return = 100(Net profit/$1,000) 


----------------------------------------------------------------------
----------
Jack Schwager is the author of the best-sellers Market Wizards and 
The New Market Wizards, as well as other works. His most recent is a 
12-tape video course, Jack Schwager's Complete Guide To Designing And 
Testing Trading Systems, which was produced with Omega Research. 
Schwager is the CEO of Wizard Trading, a CTA firm that began managing 
client funds in 1990 and is currently associated with the Phoenix-
based CTA Trendstat. His previous experience also includes 22 years 
as the director of futures research for some of Wall Street's leading 
firms.


rgds, Pal


--- In amibroker@xxxxxxxxxxxxxxx, "palsanand" <palsanand@xxxx> wrote:
> http://groups.yahoo.com/group/amibroker/message/49959
> http://groups.yahoo.com/group/amibroker/message/53334
> http://groups.yahoo.com/group/amibroker/message/53439
> 
> rgds, Pal
> --- In amibroker@xxxxxxxxxxxxxxx, "Gary A. Serkhoshian" 
> <serkhoshian777@xxxx> wrote:
> > Phsst,
> >  
> > I'll just carry forward what others have shared, but I'd love to 
> read responses from Fred, Chuck, Howard, and Mark in terms of their 
> opinions and usage if they have the time/inclination.
> >  
> > The big headline to share is on Sharpe and UPI, IMHO.  Both are 
> risk-adjusted measures of the equity curve, but look at it from 
> different perspectives.  Sharpe penalizes both positive and 
negative 
> volatility, whereas UPI penalizes only negative volatility (i.e. 
> drawdowns).  For this reason, I choose to look at UPI becuase I 
> personally don't mind positive volatility (ie. big gains in short 
> period of time).  CAR/MaxDD is a quick, back of the envelope 
> calculation that will get you in the same ballpark as UPI.
> >  
> > Also understand that money market will have the best (i.e. 
> infinity) Sharpe and UPI because you have no volatility in the 
equity 
> curve.  Of course, we are not interested 0.5% CAR so we cast aside 
> MM.  No guts, no glory.
> >  
> > Hope this helps,
> > Gary
> > 
> > Phsst <phsst@xxxx> wrote:
> > Chuck,
> > 
> > As you can see, I've changed the Subject because I think we may 
have
> > an opportunity thru your questions to get better insight into 
> reported
> > figures on the backtester report.
> > 
> > If you reply, and I certainly hope that you do, just drop your 
name
> > from the Subject.
> > 
> > You are right... there are some things very wrong with the 
backtest
> > reports that I posted, and I had not noticed them until you 
pointed
> > them out.
> > 
> > 
> > Specifically:
> > 
> > CR> 1.   Both of them show nice annual returns (89-140%).
> > 
> > Annual returns this high should be treated with suspicion and 
cause
> > the system developer to be extra vigilant in analyzing the 
reported
> > figures. Unfortunately I failed to notice the discrepancies that 
you
> > noted. You can bet that I'll be more vigilant in the future, 
looking
> > specifically at those figures you zeroed in on.
> > 
> > 
> > CR> 2.   Yet, the average p/l is negative (2-4%).
> > 
> > I thought I might have made a mistake copying/pasting on this 
one, 
> but
> > I have verified that the actual backtest report contained these 
> figures.
> > 
> > CR> 3.   The Sharpe ratio is negative.
> > 
> > I have to plead ignorance here. I've read as much as is available 
in
> > the documentation relating to the Sharpe ratio (among others). But
> > when I am unable to get a clear idea of exactly how to interpret 
> some
> > of these metrics then I tend to just ignore them on the backtest 
> report. 
> > 
> > I would really like to see a dialog on this forum related to the 
> real
> > value of Sharpe, Ulcer, etc.
> > 
> > CR> 4.   The average win is 3%.
> > 
> > Same response as #2.
> > 
> > CR> 5.   The average loss is 10-11%.
> > 
> > Same response as #2.
> > 
> > CR> What's the story???
> > 
> > Good question Chuck.
> > 
> > In addition to posting this response, I'll send TJ the actual 
> backtest
> > report(s) to see if there is a rational explanation for the 
reported
> > figures which he could post here, or if it is a genuine bug.
> > 
> > If TJ needs the code, I'll be glad to send it to him.
> > 
> > If there is a problem with my backtest AFL code, then I'd want to
> > understand what coding conditions could cause this.
> > 
> > Anyway, thanks for taking a critical look at the figures.
> > 
> > Do you suppose that you and others who have good insight into the
> > reported metrics could help the rest of us better understand how 
to
> > interpret the last eleven (11) reported figures on the Backtest 
> Report?
> > 
> > Regards,
> > 
> > Phsst
> > 
> > --- In amibroker@xxxxxxxxxxxxxxx, "Chuck Rademacher"
> > <chuck_rademacher@x> wrote:
> > > Phsst,
> > > 
> > > There seems to be something radically wrong with your stats 
> (below):
> > > 
> > > 1.   Both of them show nice annual returns (89-140%).
> > > 
> > > 2.   Yet, the average p/l is negative (2-4%).
> > > 
> > > 3.   The Sharpe ratio is negative.
> > > 
> > > 4.   The average win is 3%.
> > > 
> > > 5.   The average loss is 10-11%.
> > > 
> > > What's the story???
> > >   -----Original Message-----
> > >   From: Phsst [mailto:phsst@x...]
> > >   Sent: Saturday, December 13, 2003 5:31 PM
> > >   To: amibroker@xxxxxxxxxxxxxxx
> > >   Subject: [amibroker] Re: PositionScore Ideas
> > > 
> > > 
> > >   Greg,
> > > 
> > >   This backtest comparison is for illustrative purposes only. I 
> make no
> > >   claims regarding these test results other than the AFL and 
Setup
> > >   criteria was identical for both tests. The only difference 
was 
> the
> > >   assignment of PositionScore = QRS versus PositionScore = RSW.
> > > 
> > >   NOTE:
> > > 
> > >   // RSW = .4*(Total Return 13-Week)+.3*(Total Return 26-Week)
+.3*
> (Total
> > >   Return 1-Year)
> > >   tr13 = 0.4 * (C - Ref(C, -65)) / Ref(C, -65) * 100;
> > >   tr26 = 0.3 * (C - Ref(C, -130)) / Ref(C, -130) * 100;
> > >   tr52 = 0.3 * (C - Ref(C, -260)) / Ref(C, -260) * 100;
> > >   RSW = tr13 + tr26 + tr52;
> > >   PositionScore = RSW;
> > > 
> > >   Date Range 6/1/1995 to Present (No QRS scores exist prior to 
> this)
> > > 
> > >   Direct comparison:
> > > 
> > >         RSW SCORE            QRS SCORE
> > >         Long trades            Long trades
> > >   Initial capital      100000            100000
> > >   Ending capital      22984180      190338380
> > >   Net Profit      22884180      190238380
> > >   Net Profit %      22884.18%      190238.38%
> > >   Exposure %      94.25%            94.16%
> > >   Net RAR %      24280.98%      202035.63%
> > >   Annual Return %      89.07%              142.19%
> > >   Risk Adj Retn %      94.50%            151.01%
> > > 
> > >   All trades      7431 (100.00 %)      7487 (100.00 %)
> > >   Avg. Profit/Loss      3079.56 25409.16
> > >   Avg. Profit/Loss %      -4.16%      -2.88%
> > >   Avg. Bars Held               2.65      2.63
> > > 
> > >   Winners               3829 (51.53 %)      4066 (54.31 %)
> > >   Total Profit      64437022.92      436648089.7
> > >   Avg. Profit      16828.68      107390.09
> > >   Avg. Profit %      3.10%            3.13%
> > >   Avg. Bars Held      2.33            2.33
> > >   Max. Consecutive      17      17
> > >   Largest win      978262.15      7798920.06
> > >   # bars in largest win      2      2
> > > 
> > >   Losers      3602 (48.47 %)            3421 (45.69 %)
> > >   Total Loss      -41552842.92      -246409709.4
> > >   Avg. Loss      -11536.05      -72028.56
> > >   Avg. Loss %      -11.88%            -10.03%
> > >   Avg. Bars Held      2.98            2.99
> > >   Max. Consecutive      13      12
> > >   Largest loss      -542767            -4301835.5
> > >   # bars in largest loss      6      6
> > > 
> > >   Max. trade drawdown      -610851.92      -4864832.72
> > >   Max. trade % drawdown      -98.69%            -99.67%
> > >   Max. system drawdown      -2119041.36      -15587244.83
> > >   Max. system % drawdown      -34.68%            -27.63%
> > >   Recovery Factor      10.8            12.2
> > >   CAR/MaxDD      2.57            5.15
> > >   RAR/MaxDD      2.72            5.46
> > >   Profit Factor      1.55            1.77
> > >   Payoff Ratio      1.46            1.49
> > >   Standard Error      2982472.3      25676798.01
> > >   Risk-Reward Ratio      0.44      0.38
> > >   Ulcer Index      12.1            7.64
> > >   Ulcer Performance Index      6.92      17.9
> > >   Sharpe Ratio of trades      -0.72      -0.86
> > >   K-Ratio                       1.09      0.93
> > > 
> > >   FWIW, I have some other systems / variations that I'll run a 
> RSW vs.
> > >   QRS comparison on. If there are any notable improvements to 
the 
> RSW
> > >   results, I'll post them.
> > > 
> > >   Regards,
> > > 
> > >   Phsst
> > > 
> > >   --- In amibroker@xxxxxxxxxxxxxxx, "Greg" <gregbean@xxxx> 
wrote:
> > >   > Phsst,
> > >   >
> > >   > Yes I think it is (Total Return 13-Week) means (Pct Price 
> gain in
> > >   > 13-Weeks). The terms are from ValueLine, I think.
> > >   > http://www.valueline.com/
> > >   >
> > >   > IBD definition of Relative Strength:
> > >   >
> > >   >  Relative Price Strength (RS) Rating or Relative 
StrengthThis 
> IBD
> > >   SmartSelect® Corporate Rating measures each stock's price 
> performance
> > >   over the latest twelve months compared to all other stocks. 
The 
> rating
> > >   scale ranges from 1 (lowest) to 99 (highest). Stocks rating 
> below 70
> > >   indicate weaker or more laggard relative price performance.
> > >   > http://www.investors.com/
> > >   >
> > >   >
> > >   > Greg
> > >   >
> > >   >   ----- Original Message -----
> > >   >   From: Phsst
> > >   >   To: amibroker@xxxxxxxxxxxxxxx
> > >   >   Sent: Saturday, December 13, 2003 4:52 PM
> > >   >   Subject: [amibroker] Re: PositionScore Ideas
> > >   >
> > >   >
> > >   >   Greg,
> > >   >
> > >   >   I'll be happy to do a comparison on just about anything 
that
> > might be
> > >   >   comparable to IDB's RS Rank.
> > >   >
> > >   >   I assume that (Total Return 13-Week) means (Pct Price 
gain 
> in
> > >   >   13-Weeks), and so on?
> > >   >
> > >   >   Worth noting here, that IDB's RS Rank is a score between 
1 
> and 100
> > >   >   that ranks each particular stock against the whole mkt 
for 
> the
> > >   past year.
> > >   >
> > >   >   But for positionscore pusposes, we are not limited to a 
> score
> > of 1 -
> > >   >   100, so I can do the raw comparison of results from your 
> formula
> > >   to QRS.
> > >   >
> > >   >   I'll post back later under this same Subject.
> > >   >
> > >   >   Regards
> > >   >
> > >   >   Phsst
> > >   >
> > >   >
> > >   >   --- In amibroker@xxxxxxxxxxxxxxx, "Greg" <gregbean@xxxx> 
> wrote:
> > >   >   > Hi Phsst.
> > >   >   >
> > >   >   > Here is a formula that I have been told closely follows 
> that
> > of IBD.
> > >   >   Could you please do the comparison you offered ?
> > >   >   >
> > >   >   > RSW = .4*(Total Return 13-Week)+.3*(Total Return
> > 26-Week)+.3*(Total
> > >   >   Return 1-Year)
> > >   >   >
> > >   >   > Thanks,
> > >   >   > Greg
> > >   >   >   ----- Original Message -----
> > >   >   >   From: Phsst
> > >   >   >   To: amibroker@xxxxxxxxxxxxxxx
> > >   >   >   Sent: Saturday, December 13, 2003 1:26 PM
> > >   >   >   Subject: [amibroker] Re: PositionScore Ideas
> > >   >   >
> > >   >   >
> > >   >   >   Al,
> > >   >   >
> > >   >   >   My favorite is the QP2 QRS value (GetExtraData
("QRS"). 
> The
> > QP2 QRS
> > >   >   >   value is supposed to be a 'knockoff' of the IBD RS 
> ranking
> > score.
> > >   >   >
> > >   >   >   I almost always get a significant boost using this 
> ranking
> > >   figure as
> > >   >   >   as the positionscore.
> > >   >   >
> > >   >   >   If you do not have QP2, but have any ideas about how 
to 
> do
> > >   your own RS
> > >   >   >   Rank calculation, I'd be happy to run some 
comparisons 
> for
> > you (or
> > >   >   >   anyone else) to measure your calculated RS Rank 
against
> > QP2's QRS
> > >   >   rank.
> > >   >   >
> > >   >   >   Cheers,
> > >   >   >
> > >   >   >   Phsst
> > >   >   >   --- In amibroker@xxxxxxxxxxxxxxx, "Al Venosa" 
> <advenosa@xxxx>
> > >   wrote:
> > >   >   >   > Hi, all:
> > >   >   >   >
> > >   >   >   > I've been experimenting with variuos short term 
> trading
> > systems
> > >   >   >   lately (average trade durations of about 2.5 days), 
and 
> I was
> > >   looking
> > >   >   >   for ideas on how best to rank a watchlist to get the 
> best
> > >   candidates
> > >   >   >   for portfolio trading a basket of 4 stocks. I was 
> wondering if
> > >   anyone
> > >   >   >   would care to share any ideas on how you use the 
> PositionScore
> > >   >   >   function to rank your candidate list (using regular 
> mode, not
> > >   >   >   rotational mode). I've tried combinations of turnover 
> and
> > >   volatility,
> > >   >   >   but I'd like to try other ideas. I'm not asking 
anyone 
> to give
> > >   away
> > >   >   >   any secrets, and, yes, I am aware of TJ's example in 
the
> > help file
> > >   >   >   (PositionScore = 100 -RSI());), but I was just 
looking 
> for
> > >   more ideas.
> > >   >   >   I'm not even sure if this question is too vague or 
not. 
> If it
> > >   is, I'm
> > >   >   >   sure you'll tell me. TIA.
> > >   >   >   >
> > >   >   >   > Al Venosa
> > >   >   >   > advenosa@xxxx
> > >   >   >   >
> > >   >   >   >
> > >   >   >   > ---
> > >   >   >   > Outgoing mail is certified Virus Free.
> > >   >   >   > Checked by AVG anti-virus system 
> (http://www.grisoft.com).
> > >   >   >   > Version: 6.0.543 / Virus Database: 337 - Release 
Date:
> > >   11/21/2003
> > >   >   >
> > >   >   >
> > >   >   >         Yahoo! Groups Sponsor
> > >   >   >
> > >   >   >
> > >   >   >
> > >   >   >   Send BUG REPORTS to bugs@xxxx
> > >   >   >   Send SUGGESTIONS to suggest@xxxx
> > >   >   >   -----------------------------------------
> > >   >   >   Post AmiQuote-related messages ONLY to:
> > amiquote@xxxxxxxxxxxxxxx
> > >   >   >   (Web page: 
> http://groups.yahoo.com/group/amiquote/messages/)
> > >   >   >   --------------------------------------------
> > >   >   >   Check group FAQ at:
> > >   >   
http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > >   >   >
> > >   >   >   Your use of Yahoo! Groups is subject to the Yahoo! 
> Terms of
> > >   Service.
> > >   >
> > >   >
> > >   >         Yahoo! Groups Sponsor
> > >   >               ADVERTISEMENT
> > >   >
> > >   >
> > >   >
> > >   >
> > >   >   Send BUG REPORTS to bugs@xxxx
> > >   >   Send SUGGESTIONS to suggest@xxxx
> > >   >   -----------------------------------------
> > >   >   Post AmiQuote-related messages ONLY to: 
> amiquote@xxxxxxxxxxxxxxx
> > >   >   (Web page: 
http://groups.yahoo.com/group/amiquote/messages/)
> > >   >   --------------------------------------------
> > >   >   Check group FAQ at:
> > >   http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > >   >
> > >   >   Your use of Yahoo! Groups is subject to the Yahoo! Terms 
of
> > Service.
> > > 
> > > 
> > >         Yahoo! Groups Sponsor
> > >               ADVERTISEMENT
> > > 
> > > 
> > > 
> > > 
> > >   Send BUG REPORTS to bugs@xxxx
> > >   Send SUGGESTIONS to suggest@xxxx
> > >   -----------------------------------------
> > >   Post AmiQuote-related messages ONLY to: 
amiquote@xxxxxxxxxxxxxxx
> > >   (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > >   --------------------------------------------
> > >   Check group FAQ at:
> > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > 
> > >   Your use of Yahoo! Groups is subject to the Yahoo! Terms of 
> Service.
> > 
> > 
> > Yahoo! Groups SponsorADVERTISEMENT
> > 
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at: 
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
> > 
> > Your use of Yahoo! Groups is subject to the Yahoo! Terms of 
> Service. 
> > 
> > 
> > ---------------------------------
> > Do you Yahoo!?
> > New Yahoo! Photos - easier uploading and sharing


------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/GHeqlB/TM
---------------------------------------------------------------------~->

Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/