[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Understanding Portfolio Backtest Reports ..was..Re: PositionScore Ideas



PureBytes Links

Trading Reference Links

http://groups.yahoo.com/group/amibroker/message/49959
http://groups.yahoo.com/group/amibroker/message/53334
http://groups.yahoo.com/group/amibroker/message/53439

rgds, Pal
--- In amibroker@xxxxxxxxxxxxxxx, "Gary A. Serkhoshian" 
<serkhoshian777@xxxx> wrote:
> Phsst,
>  
> I'll just carry forward what others have shared, but I'd love to 
read responses from Fred, Chuck, Howard, and Mark in terms of their 
opinions and usage if they have the time/inclination.
>  
> The big headline to share is on Sharpe and UPI, IMHO.  Both are 
risk-adjusted measures of the equity curve, but look at it from 
different perspectives.  Sharpe penalizes both positive and negative 
volatility, whereas UPI penalizes only negative volatility (i.e. 
drawdowns).  For this reason, I choose to look at UPI becuase I 
personally don't mind positive volatility (ie. big gains in short 
period of time).  CAR/MaxDD is a quick, back of the envelope 
calculation that will get you in the same ballpark as UPI.
>  
> Also understand that money market will have the best (i.e. 
infinity) Sharpe and UPI because you have no volatility in the equity 
curve.  Of course, we are not interested 0.5% CAR so we cast aside 
MM.  No guts, no glory.
>  
> Hope this helps,
> Gary
> 
> Phsst <phsst@xxxx> wrote:
> Chuck,
> 
> As you can see, I've changed the Subject because I think we may have
> an opportunity thru your questions to get better insight into 
reported
> figures on the backtester report.
> 
> If you reply, and I certainly hope that you do, just drop your name
> from the Subject.
> 
> You are right... there are some things very wrong with the backtest
> reports that I posted, and I had not noticed them until you pointed
> them out.
> 
> 
> Specifically:
> 
> CR> 1.   Both of them show nice annual returns (89-140%).
> 
> Annual returns this high should be treated with suspicion and cause
> the system developer to be extra vigilant in analyzing the reported
> figures. Unfortunately I failed to notice the discrepancies that you
> noted. You can bet that I'll be more vigilant in the future, looking
> specifically at those figures you zeroed in on.
> 
> 
> CR> 2.   Yet, the average p/l is negative (2-4%).
> 
> I thought I might have made a mistake copying/pasting on this one, 
but
> I have verified that the actual backtest report contained these 
figures.
> 
> CR> 3.   The Sharpe ratio is negative.
> 
> I have to plead ignorance here. I've read as much as is available in
> the documentation relating to the Sharpe ratio (among others). But
> when I am unable to get a clear idea of exactly how to interpret 
some
> of these metrics then I tend to just ignore them on the backtest 
report. 
> 
> I would really like to see a dialog on this forum related to the 
real
> value of Sharpe, Ulcer, etc.
> 
> CR> 4.   The average win is 3%.
> 
> Same response as #2.
> 
> CR> 5.   The average loss is 10-11%.
> 
> Same response as #2.
> 
> CR> What's the story???
> 
> Good question Chuck.
> 
> In addition to posting this response, I'll send TJ the actual 
backtest
> report(s) to see if there is a rational explanation for the reported
> figures which he could post here, or if it is a genuine bug.
> 
> If TJ needs the code, I'll be glad to send it to him.
> 
> If there is a problem with my backtest AFL code, then I'd want to
> understand what coding conditions could cause this.
> 
> Anyway, thanks for taking a critical look at the figures.
> 
> Do you suppose that you and others who have good insight into the
> reported metrics could help the rest of us better understand how to
> interpret the last eleven (11) reported figures on the Backtest 
Report?
> 
> Regards,
> 
> Phsst
> 
> --- In amibroker@xxxxxxxxxxxxxxx, "Chuck Rademacher"
> <chuck_rademacher@x> wrote:
> > Phsst,
> > 
> > There seems to be something radically wrong with your stats 
(below):
> > 
> > 1.   Both of them show nice annual returns (89-140%).
> > 
> > 2.   Yet, the average p/l is negative (2-4%).
> > 
> > 3.   The Sharpe ratio is negative.
> > 
> > 4.   The average win is 3%.
> > 
> > 5.   The average loss is 10-11%.
> > 
> > What's the story???
> >   -----Original Message-----
> >   From: Phsst [mailto:phsst@x...]
> >   Sent: Saturday, December 13, 2003 5:31 PM
> >   To: amibroker@xxxxxxxxxxxxxxx
> >   Subject: [amibroker] Re: PositionScore Ideas
> > 
> > 
> >   Greg,
> > 
> >   This backtest comparison is for illustrative purposes only. I 
make no
> >   claims regarding these test results other than the AFL and Setup
> >   criteria was identical for both tests. The only difference was 
the
> >   assignment of PositionScore = QRS versus PositionScore = RSW.
> > 
> >   NOTE:
> > 
> >   // RSW = .4*(Total Return 13-Week)+.3*(Total Return 26-Week)+.3*
(Total
> >   Return 1-Year)
> >   tr13 = 0.4 * (C - Ref(C, -65)) / Ref(C, -65) * 100;
> >   tr26 = 0.3 * (C - Ref(C, -130)) / Ref(C, -130) * 100;
> >   tr52 = 0.3 * (C - Ref(C, -260)) / Ref(C, -260) * 100;
> >   RSW = tr13 + tr26 + tr52;
> >   PositionScore = RSW;
> > 
> >   Date Range 6/1/1995 to Present (No QRS scores exist prior to 
this)
> > 
> >   Direct comparison:
> > 
> >         RSW SCORE            QRS SCORE
> >         Long trades            Long trades
> >   Initial capital      100000            100000
> >   Ending capital      22984180      190338380
> >   Net Profit      22884180      190238380
> >   Net Profit %      22884.18%      190238.38%
> >   Exposure %      94.25%            94.16%
> >   Net RAR %      24280.98%      202035.63%
> >   Annual Return %      89.07%              142.19%
> >   Risk Adj Retn %      94.50%            151.01%
> > 
> >   All trades      7431 (100.00 %)      7487 (100.00 %)
> >   Avg. Profit/Loss      3079.56 25409.16
> >   Avg. Profit/Loss %      -4.16%      -2.88%
> >   Avg. Bars Held               2.65      2.63
> > 
> >   Winners               3829 (51.53 %)      4066 (54.31 %)
> >   Total Profit      64437022.92      436648089.7
> >   Avg. Profit      16828.68      107390.09
> >   Avg. Profit %      3.10%            3.13%
> >   Avg. Bars Held      2.33            2.33
> >   Max. Consecutive      17      17
> >   Largest win      978262.15      7798920.06
> >   # bars in largest win      2      2
> > 
> >   Losers      3602 (48.47 %)            3421 (45.69 %)
> >   Total Loss      -41552842.92      -246409709.4
> >   Avg. Loss      -11536.05      -72028.56
> >   Avg. Loss %      -11.88%            -10.03%
> >   Avg. Bars Held      2.98            2.99
> >   Max. Consecutive      13      12
> >   Largest loss      -542767            -4301835.5
> >   # bars in largest loss      6      6
> > 
> >   Max. trade drawdown      -610851.92      -4864832.72
> >   Max. trade % drawdown      -98.69%            -99.67%
> >   Max. system drawdown      -2119041.36      -15587244.83
> >   Max. system % drawdown      -34.68%            -27.63%
> >   Recovery Factor      10.8            12.2
> >   CAR/MaxDD      2.57            5.15
> >   RAR/MaxDD      2.72            5.46
> >   Profit Factor      1.55            1.77
> >   Payoff Ratio      1.46            1.49
> >   Standard Error      2982472.3      25676798.01
> >   Risk-Reward Ratio      0.44      0.38
> >   Ulcer Index      12.1            7.64
> >   Ulcer Performance Index      6.92      17.9
> >   Sharpe Ratio of trades      -0.72      -0.86
> >   K-Ratio                       1.09      0.93
> > 
> >   FWIW, I have some other systems / variations that I'll run a 
RSW vs.
> >   QRS comparison on. If there are any notable improvements to the 
RSW
> >   results, I'll post them.
> > 
> >   Regards,
> > 
> >   Phsst
> > 
> >   --- In amibroker@xxxxxxxxxxxxxxx, "Greg" <gregbean@xxxx> wrote:
> >   > Phsst,
> >   >
> >   > Yes I think it is (Total Return 13-Week) means (Pct Price 
gain in
> >   > 13-Weeks). The terms are from ValueLine, I think.
> >   > http://www.valueline.com/
> >   >
> >   > IBD definition of Relative Strength:
> >   >
> >   >  Relative Price Strength (RS) Rating or Relative StrengthThis 
IBD
> >   SmartSelect® Corporate Rating measures each stock's price 
performance
> >   over the latest twelve months compared to all other stocks. The 
rating
> >   scale ranges from 1 (lowest) to 99 (highest). Stocks rating 
below 70
> >   indicate weaker or more laggard relative price performance.
> >   > http://www.investors.com/
> >   >
> >   >
> >   > Greg
> >   >
> >   >   ----- Original Message -----
> >   >   From: Phsst
> >   >   To: amibroker@xxxxxxxxxxxxxxx
> >   >   Sent: Saturday, December 13, 2003 4:52 PM
> >   >   Subject: [amibroker] Re: PositionScore Ideas
> >   >
> >   >
> >   >   Greg,
> >   >
> >   >   I'll be happy to do a comparison on just about anything that
> might be
> >   >   comparable to IDB's RS Rank.
> >   >
> >   >   I assume that (Total Return 13-Week) means (Pct Price gain 
in
> >   >   13-Weeks), and so on?
> >   >
> >   >   Worth noting here, that IDB's RS Rank is a score between 1 
and 100
> >   >   that ranks each particular stock against the whole mkt for 
the
> >   past year.
> >   >
> >   >   But for positionscore pusposes, we are not limited to a 
score
> of 1 -
> >   >   100, so I can do the raw comparison of results from your 
formula
> >   to QRS.
> >   >
> >   >   I'll post back later under this same Subject.
> >   >
> >   >   Regards
> >   >
> >   >   Phsst
> >   >
> >   >
> >   >   --- In amibroker@xxxxxxxxxxxxxxx, "Greg" <gregbean@xxxx> 
wrote:
> >   >   > Hi Phsst.
> >   >   >
> >   >   > Here is a formula that I have been told closely follows 
that
> of IBD.
> >   >   Could you please do the comparison you offered ?
> >   >   >
> >   >   > RSW = .4*(Total Return 13-Week)+.3*(Total Return
> 26-Week)+.3*(Total
> >   >   Return 1-Year)
> >   >   >
> >   >   > Thanks,
> >   >   > Greg
> >   >   >   ----- Original Message -----
> >   >   >   From: Phsst
> >   >   >   To: amibroker@xxxxxxxxxxxxxxx
> >   >   >   Sent: Saturday, December 13, 2003 1:26 PM
> >   >   >   Subject: [amibroker] Re: PositionScore Ideas
> >   >   >
> >   >   >
> >   >   >   Al,
> >   >   >
> >   >   >   My favorite is the QP2 QRS value (GetExtraData("QRS"). 
The
> QP2 QRS
> >   >   >   value is supposed to be a 'knockoff' of the IBD RS 
ranking
> score.
> >   >   >
> >   >   >   I almost always get a significant boost using this 
ranking
> >   figure as
> >   >   >   as the positionscore.
> >   >   >
> >   >   >   If you do not have QP2, but have any ideas about how to 
do
> >   your own RS
> >   >   >   Rank calculation, I'd be happy to run some comparisons 
for
> you (or
> >   >   >   anyone else) to measure your calculated RS Rank against
> QP2's QRS
> >   >   rank.
> >   >   >
> >   >   >   Cheers,
> >   >   >
> >   >   >   Phsst
> >   >   >   --- In amibroker@xxxxxxxxxxxxxxx, "Al Venosa" 
<advenosa@xxxx>
> >   wrote:
> >   >   >   > Hi, all:
> >   >   >   >
> >   >   >   > I've been experimenting with variuos short term 
trading
> systems
> >   >   >   lately (average trade durations of about 2.5 days), and 
I was
> >   looking
> >   >   >   for ideas on how best to rank a watchlist to get the 
best
> >   candidates
> >   >   >   for portfolio trading a basket of 4 stocks. I was 
wondering if
> >   anyone
> >   >   >   would care to share any ideas on how you use the 
PositionScore
> >   >   >   function to rank your candidate list (using regular 
mode, not
> >   >   >   rotational mode). I've tried combinations of turnover 
and
> >   volatility,
> >   >   >   but I'd like to try other ideas. I'm not asking anyone 
to give
> >   away
> >   >   >   any secrets, and, yes, I am aware of TJ's example in the
> help file
> >   >   >   (PositionScore = 100 -RSI());), but I was just looking 
for
> >   more ideas.
> >   >   >   I'm not even sure if this question is too vague or not. 
If it
> >   is, I'm
> >   >   >   sure you'll tell me. TIA.
> >   >   >   >
> >   >   >   > Al Venosa
> >   >   >   > advenosa@xxxx
> >   >   >   >
> >   >   >   >
> >   >   >   > ---
> >   >   >   > Outgoing mail is certified Virus Free.
> >   >   >   > Checked by AVG anti-virus system 
(http://www.grisoft.com).
> >   >   >   > Version: 6.0.543 / Virus Database: 337 - Release Date:
> >   11/21/2003
> >   >   >
> >   >   >
> >   >   >         Yahoo! Groups Sponsor
> >   >   >
> >   >   >
> >   >   >
> >   >   >   Send BUG REPORTS to bugs@xxxx
> >   >   >   Send SUGGESTIONS to suggest@xxxx
> >   >   >   -----------------------------------------
> >   >   >   Post AmiQuote-related messages ONLY to:
> amiquote@xxxxxxxxxxxxxxx
> >   >   >   (Web page: 
http://groups.yahoo.com/group/amiquote/messages/)
> >   >   >   --------------------------------------------
> >   >   >   Check group FAQ at:
> >   >   http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >   >   >
> >   >   >   Your use of Yahoo! Groups is subject to the Yahoo! 
Terms of
> >   Service.
> >   >
> >   >
> >   >         Yahoo! Groups Sponsor
> >   >               ADVERTISEMENT
> >   >
> >   >
> >   >
> >   >
> >   >   Send BUG REPORTS to bugs@xxxx
> >   >   Send SUGGESTIONS to suggest@xxxx
> >   >   -----------------------------------------
> >   >   Post AmiQuote-related messages ONLY to: 
amiquote@xxxxxxxxxxxxxxx
> >   >   (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> >   >   --------------------------------------------
> >   >   Check group FAQ at:
> >   http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >   >
> >   >   Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> Service.
> > 
> > 
> >         Yahoo! Groups Sponsor
> >               ADVERTISEMENT
> > 
> > 
> > 
> > 
> >   Send BUG REPORTS to bugs@xxxx
> >   Send SUGGESTIONS to suggest@xxxx
> >   -----------------------------------------
> >   Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> >   (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> >   --------------------------------------------
> >   Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > 
> >   Your use of Yahoo! Groups is subject to the Yahoo! Terms of 
Service.
> 
> 
> Yahoo! Groups SponsorADVERTISEMENT
> 
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at: 
http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
> 
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of 
Service. 
> 
> 
> ---------------------------------
> Do you Yahoo!?
> New Yahoo! Photos - easier uploading and sharing


------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/GHeqlB/TM
---------------------------------------------------------------------~->

Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/