|
PureBytes Links
Trading Reference Links
|
--- In amibroker@xxxxxxxxxxxxxxx, "Ken Close" <closeks@xxxx> wrote:
> Please share your workaround and other thoughts
> and ideas you have about how to evaluate multiple
> simultaneous positions.
Ken, Steve, and Chuck:
My workarounds involve varying degrees of extra "work". I have not
put a lot of time into refining such workaround because they will
all be obsolete when Tomasz adds portfolio testing to AB.
First some background. In its present state, AB does a great job
backtesting and optimizing SINGLE stocks. However, when backtesting
and optimizing groups of stocks various "issues" arise. There may be
additional issues, but I am aware of the following:
A - The bais problem arises because there will be more trades in
soee time periods than others -- sometimes a lot more. When one time
period gives a lot more trades than another time period, the summary
results will be biased to the period with the most trades.
B - One implication of A, is that earlier years will in general be
given less weight in the final results than more recent years. This
is for 3 reasons. First there are more stocks being traded today.
Second, databases with survivorship bias will give further weight to
more recent years. Third, minimum average volume filters will
exclude a greater percentage of stocks 12 years ago than 1 or 2
years ago (trading volume has increased).
C - Another implication is a bit harder to explain. Let's say one is
testing a strategy that will pick 10 or 15 stocks in an average year
and these picks tend to loose money, but once every 3 or 4 years
market conditions are such that the strategy picks 100 stocks all at
once and on this and only this time the strategy produes big
winners. The summary back test results will suggest you have a real
winner when you really are looking at a strategy that can be
expected to loose most of the time.
D - The opposite of C is also possible. A strategy that may only
pick a dozen stocks most times and these are over all winners may be
rejected if only time in the test range, the strategy picks 150
stocks but this is the one time the strategy gives a significant
drawdown.
E - If one attempts to use the AddToComposite function, special care
is needed so it will not make a loosing system appear to be a winner
just because new stocks are added to the Composite each year.
F - Optimize a market timing method with a basket of stocks is, in
my opinion, very prone to giving curve fit results. The reason is
that the optimization just has to find 1 or 2 times periods that
generates a lot of winning trades which can mask the fact the system
generates losses most of the time. Extra diligence is needed when
evaluating optimization results. Some of the usual methods for
detecting curve fitting will not work (such as scanning a summary
list of optimization results to see if there are loss canyons -- the
loss canyons might be there but be masked by dominance of a mulitude
of winning trades for one signal period).
OK, enough back ground on the underlying issue (a varying number of
trades for each signal) and possible problems that come from it.
What are some work arounds? There is a quick workaround and a slow
one. The slow one is best but it is -- slow, so I use the quick one
more often.
1. A Quick Work Around. The purpose of this work around is to reduce
the possibility of a large group of trades in a single time
dominating the results. I use the Optimization feature to group
results into years. For a long system, I just add a statement like
this:
TESTyear = Optimize("TESTyear", 2000, 1985, 2003, 1);
OKyear = TESTyear == YEAR();
BUY = OKyear AND ...your buy code...
1a. Note: If you use Exit and Reverse systems, you can not use
COVER = BUY; //because some trades will never exit if the trades
normal exit signal comes in the next calendar year.
1b. What one wants to see is a system that is profitable for every
year in the optimization results list.
1c. One could, although I have not, make shorter periods than 1 year
by using a second optimization line using MONTHS.
2. The slower but best work around is to have AB do a back test and
then export the entire trade list to Excel. Use Excel's Pivot Table
feature to group the trades by entry date of the trade. This assumes
the system being tested is based on a market timing signal so that
all the trades in one batch will have the same entry date. In the
data section of the table, set the results to average (rather than
sum which is the default I think).
2a. I prefer to put AB's "percent profit" figure into the data
section of Excel's pivot table, but you might prefer to focus
another item. Use "profit" with care since the actual amount will
vary from stock to stock in the later periods due to compounding
(assuming the system is profitable).
2b. One major headache is caused by "stray" stocks that enter a
trade one day behind most of the other stocks (again I am talking
about systems that have entries based on market timing signals). I
expect this one day late entry is due to how AB fills data wholes in
individual stocks since it even occurs when I use the CROSS function
to make sure there is only 1 entry date for each market timing
cycle. Whatever the reason, even just a few "stray" trade entries
really mess up the Excel pivot table. Each of the stray date stocks
will get its own row on the pivot table and thus a single stray will
appear to have equal weight to 20 or 50 stocks that all have the
same entry date.
2c. To get detect these strays, I add a new column to Excel before
making the pivot table. This column compares the entry date of a
stock with that of the previous stock and puts a flag in if they
differ. Then I scan the column and delete any stocks that are
strays. As you can imagine, this is time consuming, so I only do it
for the most interesting strategies.
3. Other ideas I have considered but not yet attempted. The reason I
have pursued these is my expectation that Tomasz will be adding
portfolio testing to AB in the future (hopefully near future).
3a. One could write a short program in VB, C, or Delphi that would
wead out the stray trades before importing into Excel. There might
even be a way to write an Excel macro to do this (but I am not an
expert with Excel).
3b. There is another way that has recently come to mind (as yet
untried). First create a composite based on the market timing signal
(with "1" being given to signal change days and "0" to all other
days). Long entry could be in Open; long exit in High, short entry
in Low and Short cover in Close, or separate composites could be
used. Then using the CUM() function one could number each timing
signal and with an appropriate Optimize statement, one would end up
with trades that could be sorted in the Excel pivot table not by
date of entry but by the signal number (the optimization number).
Thus a stock that entered one day later than most would have the
same Optimization number as those that entered on the "right" day
and the Excel pivot tables would not get messed up by the stray date
entries.
Just writing this up makes me tried. I look forward to the day when
portfolio level testing gets added to AB.
b
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|