[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [amibroker] Why portfolio backtester does not consider all buy signals?

PureBytes Links

Trading Reference Links
TJ:
Thanks for your kind reply and references to many advanced features of AB. I will spend more time to learn the new and powerful low level backtest features.
In the meantime, I do have some comments. (see below)
----- Original Message -----
Sent: Sunday, April 23, 2006 3:26 PM
Subject: Re: [amibroker] Why portfolio backtester does not consider all buy signals?

Hello,
 
I explained already that TRADES (not signals) are picked according to position score.
 
This is how regular backtest is designed to work and it operates as designed.
It is designed mainly because of consistency - to maintain same entry/exit points if you run single security backtest
(with your approach people would come up with "bug" reports every minute saying
that their entry dates change all the time even if they dont change the system).
 
OK. I understand that you need to maintain backward consistency since the portfolio backtest was evolved from the old individul backtester.
May I suggest that a setting be added to turn on/off "removing redundant signals" for the portfolio backtesting?
The default is "on", so that it is consistent with what it is now. This would be a great help for those of us just converted from Wealth-lab.
 
There are other reasons such as the fact that practice shows that delayed signals are very often very poor performers
so it is better to skip trade instead of entering it too late.
This may be true to many systems. But certainly not all of them. My most profitable system happens to do better with later signals. It buys dips and later signals are more likely to bounce back. IMO, a backtest software should be neutral to trading systems (i.e. not to prefer a particular  kind of practice)
 
I also don't agree with your arguments, specifically that about any limitations you see.
There are no limitations. You can code every imaginable system.
I am with you here :-) AB is more powerful and faster than all other trading softwares I had used. The limitation I refer to is that the high level backtester always removes signals it thinks redundant, but may not be redundant in real trading, and there is no way to disable that.
 
I have no doubt that by taking advantage of the low level features, I can accomplish what I want to do. However, it would a GREAT FEATURE if the setting I just mentioned can be added to disable removing redundant signals.
 
What you trying to do is perfectly doable in current AmiBroker using custom backtester.
 
There is also a separate rotational trading mode that does not remove redundant signals
(in fact it is not based on signals, but purely on score) and you can find it here:
http://www.amibroker.com/f?enablerotationaltrading
 
Thanks for your suggestion. I have tried to use rotational trading to simulate my system and have been successful to a great extent, though not 100%.
Two things have prevented me from getting 100%
1. n-bar stops are not taken when scoreNoRotate is true.
2. scoreNoRotate is global, not individual.
 
So actually AMiBroker is WAY more powerful and flexible that the other software you are quoting
because it does not only offers 2 built-in modes of backtesting (compared to just one 1 in the other soft)
but also allows to build customized backtesters that will behave  in any way you can dream of.
 
I TOTALLY AGREE WITH YOU HERE. As I converted more and more scripts from WL, I started to appreciate the great features of AB, although I still have a long way to go with low level stuff. And the speed and support for AB is PEERLESS!

Best regards,
Tomasz Janeczko
amibroker.com
----- Original Message -----
From: Mark H
Sent: Sunday, April 23, 2006 8:40 PM
Subject: Re: [amibroker] Why portfolio backtester does not consider all buy signals?

TJ:
 
Thanks for your reply.
Based on the formation I got from the links, I found that the PORTFOLIO backtester does have some limitations which may or may not be classified as design fault.
 
Your statement:
"TRADE is a buy-sell or short-cover pair.
All REDUNDANT signals are removed."
is valid only for INDIVIDUAL backtesting only, I believe.
 
Some signals considered REDUNDANT in individual backtesting may NOT be REDUNDANT as all for portfolio backtesting.
 
Let me explain what I meant:
For example, consider stocks ABC, BCD, CDE for a system taking only one trade every time.
The read lines are the trades should happen in real trading.
Stock ABC:
1/3/2005 Buy signal    PositionScore=80   
1/4/2005 Buy signal    PositionScore=100         (AmiBroker backtester removed as redundant)
1/7/2005 Buy signal    PositionScore=160         (AmiBroker backtester removed as redundant, BUT IT IS NOT)
1/10/2005 Sell signal
Stock BCD:
1/3/2005 Buy signal    PositionScore=120   
1/5/2005 Sell signal   
Stock CDE:
1/7/2005 Buy signal    PositionScore=100         (AmiBroker backtester takes this signal, but it should not)
1/10/2005 Sell signal
 
IMO, it is the responsibility of AFL code author to write code to remove redundant signals, NOT the backtester itself. The backtester should consider all signals.
I came from Wealth-lab, and in the process of porting all my scripts to AmiBroker. That's how Wealth-lab simulator behaves. I am not saying WL's way is better. But in real trading, that's the way to process signals.
 
Thanks,
 
- Mark
----- Original Message -----
Sent: Sunday, April 23, 2006 1:23 PM
Subject: Re: [amibroker] Why portfolio backtester does not consider all buy signals?

Hello,
 
There is no fault here - this is exactly as it is supposed to work. 
Backtester picks TRADES based on position score.
TRADE is a buy-sell or short-cover pair.
All REDUNDANT signals are removed.
 
Take a very close look at this chart that explains how backtester works
 
For more information.

Best regards,
Tomasz Janeczko
amibroker.com
----- Original Message -----
From: Mark H
Sent: Sunday, April 23, 2006 6:56 PM
Subject: [amibroker] Why portfolio backtester does not consider all buy signals?

Hello:
 
Maybe somebody can explain this to me. I believe I must have missed something, otherwise this would be a critical fault in the backtester.
 
I backtested a simple dip buying system against a portfolio of all common stocks (using QP3 quotes) for the month of 1/2005. I turned on the detailed log so that I can examine how it would trade. Then I ran an exploration to get a list of signals on date 1/4/05. When I compared the signals from exploration and those showed in the detailed log (for date 1/5/05, since there was one day delay), I found that the detailed log didn't take all the signals with the highest position scores.
 
I did similar comparison with other systems, and had the same observation. I have not found any setting that can force backtest to consider all signals.
 
System:

SetOption("InitialEquity", 30000);

SetOption("CommissionMode", 1); //$$ per trade

SetOption("CommissionAmount", 0.3);

SetOption("MarginRequirement", 100);

SetOption("UsePrevBarEquityForPosSizing", True);

SetTradeDelays( 1, 1, 1, 1 );

RoundLotSize = 5;

Sell = (C>=Ref(C, -1)) AND (Ref(C,-1)>=Ref(C, -2));

Buy = (C<Ref(C, -1)) AND (Ref(C, -1) < Ref(C, -2));

SetPositionSize(10, spsPercentOfEquity);

PositionScore = 100 - ROC(C, 4);

Filter = Buy OR Sell;

AddColumn( IIf( Buy, Asc("B"),Asc("S") ), "Signal", formatChar );

AddColumn(C, "Close");

AddColumn(MA(V,50)*0.05, "MA50 Vol 5%");

AddColumn(IIf(Buy, PositionScore, 0), "PositionScore", 1.5);

Detailed Log: (I run a backtest for the month of January 2005).
1/4/2005,,,,,,,,,,,,,,,,,,,,,,
,Entry signals(score):APT=Buy(112.919), TOPT=Buy(111.713), ITCD=Buy(111.429), BED=Buy(111.404), SUPG=Buy(111.079), CNTY=Buy(111.029), RUBO=Buy(110.867), IVN=Buy(109.934), JMAR=Buy(109.929), NAVI=Buy(109.926), KEYW=Buy(109.848), PARS=Buy(109.804), M...,,,,,,,,,
,Exit signals:,,,,,,,,,,,,,,,,,,,,,
,10 Open Positions: , AZL (+530), , LTRX (+2675), , RTK (+1345), , TGC (+11535), , SCON (+215), , FISI (+125), , HDSN (+2650), , BITS (+1125), , AVSR (+2235), , ADAM (+740), Equity: 28768.5, Cash: 46.8594
1/5/2005,,,,,,,,,,,,,,,,,,,,,,
,Entry signals(score):ENG=Buy(122.257), MBAY=Buy(120.382), TIV=Buy(120.055), MFRI=Buy(119.157), ANLT=Buy(118.644), KKD=Buy(118.125), VLTR=Buy(118.099), NVTL=Buy(117.766), WPSC=Buy(117.058), IAO=Buy(116.667), ACU=Buy(116.125), AKS=Buy(116.098), TELO...,,,,,,,,,
,Exit signals:ADAM=Sell, ADSX=Sell, APOL=Sell, ATLO=Sell, AVSR=Sell, BCRX=Sell, BFAM=Sell, BKSC=Sell, CAC=Sell, CCOW=Sell, CDMS=Sell, CELL=Sell, CG=Sell, CGN=Sell, CHEV=Sell, CITP=Sell, CNAF=Sell, CSWC=Sell, CTZN=Sell, DAIO=Sell, DEAR=Sell, DEBS=Se...
,Exit Long, LTRX, Price: 1.15, (Avg. exit pr. 1.15), Shares: 2675, Commission: 9.22875, (Total comm.: 18.2168), Profit: 62.0332 (2.07 %), Entry rank:115.126, Equity: 28237.2, Fx rate: 1,,,,,,,,,,,
,Exit Long, HDSN, Price: 1.11, (Avg. exit pr. 1.11), Shares: 2650, Commission: 8.8245, (Total comm.: 17.808), Profit: -70.808 (-2.36 %), Entry rank:111.765, Equity: 28237.2, Fx rate: 1,,,,,,,,,,,
,Exit Long, AVSR, Price: 1.45, (Avg. exit pr. 1.45), Shares: 2235, Commission: 9.72225, (Total comm.: 18.707), Profit: 227.143 (7.58 %), Entry rank:111.333, Equity: 28237.2, Fx rate: 1,,,,,,,,,,,
,Exit Long, ADAM, Price: 4.1, (Avg. exit pr. 4.1), Shares: 740, Commission: 9.102, (Total comm.: 18.093), Profit: 18.9068 (0.63 %), Entry rank:110.962, Equity: 28237.2, Fx rate: 1,,,,,,,,,,,
,Enter Long, ENG, Price: 2.41, Shares: 1190, Commission: 8.6037, Rank: 122.257, Equity 28220, Margin Loan: 0, Fx rate: 1,,,,,,,,,,,,,
,Enter Long, MBAY, Price: 7.49999, Shares: 380, Commission: 8.54998, Rank: 120.382, Equity 28202.9, Margin Loan: 0, Fx rate: 1,,,,,,,,,,,,,
,Enter Long, TIV, Price: 8.25, Shares: 345, Commission: 8.53875, Rank: 120.055, Equity 28185.8, Margin Loan: 0, Fx rate: 1,,,,,,,,,,,,,
,Enter Long, MFRI, Price: 9.1, Shares: 315, Commission: 8.5995, Rank: 119.157, Equity 28168.6, Margin Loan: 0, Fx rate: 1,,,,,,,,,,,,,
,10 Open Positions: , AZL (+530), , RTK (+1345), , TGC (+11535), , SCON (+215), , FISI (+125), , BITS (+1125), , ENG (+1190), , MBAY (+380), , TIV (+345), , MFRI (+315), Equity: 28289.1, Cash: 837.545
 
Exploration Results sorted by position score: (the ones in pink were taken by backtester)
Ticker,Date/Time,Signal,Close,MA50 Vol 5%,PositionScore,
AZL,1/4/2005,B,4.20,2117.90,132.69231
CLWT,1/4/2005,B,4.12,20596.03,126.29697
OLAB,1/4/2005,B,3.10,5782.43,124.18093
AGIX,1/4/2005,B,18.24,68695.09,123.71393
RTK,1/4/2005,B,1.86,20343.30,122.50000
ENG,1/4/2005,B,2.48,4869.30,122.25705
MBAY,1/4/2005,B,7.50,13010.61,120.38217
BDY,1/4/2005,B,15.75,20346.20,120.17233
TIV,1/4/2005,B,8.73,5775.60,120.05495
MFRI,1/4/2005,B,8.82,3541.42,119.15674
ABIX,1/4/2005,B,9.00,24999.93,118.91892
ANLT,1/4/2005,B,2.88,59164.97,118.64407
ADBL,1/4/2005,B,22.15,35483.04,118.56618
KKD,1/4/2005,B,10.48,123253.60,118.12501
CTG,1/4/2005,B,4.75,2408.90,118.10345
VLTR,1/4/2005,B,17.92,15662.28,118.09872
NVTL,1/4/2005,B,16.20,84408.39,117.76649
SMSI,1/4/2005,B,8.08,33207.16,117.71894
KEYW,1/4/2005,B,16.96,17987.54,117.54984
VLLY,1/4/2005,B,35.80,1717.97,117.32101
AFOP,1/4/2005,B,1.26,5090.44,117.10526
WPSC,1/4/2005,B,33.21,5254.02,117.05795
DHB,1/4/2005,B,16.53,112250.30,116.80926
SUPG,1/4/2005,B,5.84,52530.34,116.69044
IAO,1/4/2005,B,0.35,7682.40,116.66666
............
 
 
 


Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com

For other support material please check also:
http://www.amibroker.com/support.html


YAHOO! GROUPS LINKS