|
PureBytes Links
Trading Reference Links
|
B: what a great response with lots of things to think about.
About all I can say at this moment, without further reading and careful
study of all that you have written, is that I do not think I saw you
refer to the following situation:
Say you have a system but it generates different entry and exit signals
for individual stocks. There is not one market timing signal with the
associated task of managing many stocks all entering or exiting to the
tune of the market signal.
Rather, each stock generates its own signals and you enter and exit as
you wish.
If you have two stocks in your basket, it is easy to inspect and take
the trades. If you have 50 or 75 stocks, any number of which can have
signals on any day, then you have a much messier portfolio management
problem.
Do you take 5 out of 15 signals on a date? And which of the 15 do you
take?
Once you take 5 what happens the next day when there are new signals on
different stocks? OK, you specify a "hold 10 max" stocks. Therefore,
on the second day you could take 5 additional trades (you already hold
5). Which new 5?
Next day, more signals...now you can not take any of these until one of
these 10 that you hold then sell and get removed from the portfolio.
Say 2 are sold, so now you have room to take 2 more. And so it goes.
I have played with the PivotTable and I do not think arranging by dates
is enough to manage the above situation...at least I have not found the
way.
Will Tomasz' future work take this into account? Perhaps so, perhaps
not?
And finally what do you do today (assuming you have such a system
delivering nice juicy stock picks each day)? Well, you manage it by
intuition with little chance of having any kind of backtested results.
At least I think that is the case.
Thanks for taking the time to write your thoughts. I hope other join in
this discussion to broaden my horizons as to what might be possible.
Ken
-----Original Message-----
From: b519b [mailto:b519b@xxxxxxxxx]
Sent: Thursday, March 13, 2003 11:22 PM
To: amibroker@xxxxxxxxxxxxxxx
Subject: [amibroker] Work Arounds until AB gets Portfolio Testing
--- In amibroker@xxxxxxxxxxxxxxx, "Ken Close" <closeks@xxxx> wrote:
> Please share your workaround and other thoughts
> and ideas you have about how to evaluate multiple
> simultaneous positions.
Ken, Steve, and Chuck:
My workarounds involve varying degrees of extra "work". I have not
put a lot of time into refining such workaround because they will
all be obsolete when Tomasz adds portfolio testing to AB.
First some background. In its present state, AB does a great job
backtesting and optimizing SINGLE stocks. However, when backtesting
and optimizing groups of stocks various "issues" arise. There may be
additional issues, but I am aware of the following:
A - The bais problem arises because there will be more trades in
soee time periods than others -- sometimes a lot more. When one time
period gives a lot more trades than another time period, the summary
results will be biased to the period with the most trades.
B - One implication of A, is that earlier years will in general be
given less weight in the final results than more recent years. This
is for 3 reasons. First there are more stocks being traded today.
Second, databases with survivorship bias will give further weight to
more recent years. Third, minimum average volume filters will
exclude a greater percentage of stocks 12 years ago than 1 or 2
years ago (trading volume has increased).
C - Another implication is a bit harder to explain. Let's say one is
testing a strategy that will pick 10 or 15 stocks in an average year
and these picks tend to loose money, but once every 3 or 4 years
market conditions are such that the strategy picks 100 stocks all at
once and on this and only this time the strategy produes big
winners. The summary back test results will suggest you have a real
winner when you really are looking at a strategy that can be
expected to loose most of the time.
D - The opposite of C is also possible. A strategy that may only
pick a dozen stocks most times and these are over all winners may be
rejected if only time in the test range, the strategy picks 150
stocks but this is the one time the strategy gives a significant
drawdown.
E - If one attempts to use the AddToComposite function, special care
is needed so it will not make a loosing system appear to be a winner
just because new stocks are added to the Composite each year.
F - Optimize a market timing method with a basket of stocks is, in
my opinion, very prone to giving curve fit results. The reason is
that the optimization just has to find 1 or 2 times periods that
generates a lot of winning trades which can mask the fact the system
generates losses most of the time. Extra diligence is needed when
evaluating optimization results. Some of the usual methods for
detecting curve fitting will not work (such as scanning a summary
list of optimization results to see if there are loss canyons -- the
loss canyons might be there but be masked by dominance of a mulitude
of winning trades for one signal period).
OK, enough back ground on the underlying issue (a varying number of
trades for each signal) and possible problems that come from it.
What are some work arounds? There is a quick workaround and a slow
one. The slow one is best but it is -- slow, so I use the quick one
more often.
1. A Quick Work Around. The purpose of this work around is to reduce
the possibility of a large group of trades in a single time
dominating the results. I use the Optimization feature to group
results into years. For a long system, I just add a statement like
this:
TESTyear = Optimize("TESTyear", 2000, 1985, 2003, 1);
OKyear = TESTyear == YEAR();
BUY = OKyear AND ...your buy code...
1a. Note: If you use Exit and Reverse systems, you can not use
COVER = BUY; //because some trades will never exit if the trades
normal exit signal comes in the next calendar year.
1b. What one wants to see is a system that is profitable for every
year in the optimization results list.
1c. One could, although I have not, make shorter periods than 1 year
by using a second optimization line using MONTHS.
2. The slower but best work around is to have AB do a back test and
then export the entire trade list to Excel. Use Excel's Pivot Table
feature to group the trades by entry date of the trade. This assumes
the system being tested is based on a market timing signal so that
all the trades in one batch will have the same entry date. In the
data section of the table, set the results to average (rather than
sum which is the default I think).
2a. I prefer to put AB's "percent profit" figure into the data
section of Excel's pivot table, but you might prefer to focus
another item. Use "profit" with care since the actual amount will
vary from stock to stock in the later periods due to compounding
(assuming the system is profitable).
2b. One major headache is caused by "stray" stocks that enter a
trade one day behind most of the other stocks (again I am talking
about systems that have entries based on market timing signals). I
expect this one day late entry is due to how AB fills data wholes in
individual stocks since it even occurs when I use the CROSS function
to make sure there is only 1 entry date for each market timing
cycle. Whatever the reason, even just a few "stray" trade entries
really mess up the Excel pivot table. Each of the stray date stocks
will get its own row on the pivot table and thus a single stray will
appear to have equal weight to 20 or 50 stocks that all have the
same entry date.
2c. To get detect these strays, I add a new column to Excel before
making the pivot table. This column compares the entry date of a
stock with that of the previous stock and puts a flag in if they
differ. Then I scan the column and delete any stocks that are
strays. As you can imagine, this is time consuming, so I only do it
for the most interesting strategies.
3. Other ideas I have considered but not yet attempted. The reason I
have pursued these is my expectation that Tomasz will be adding
portfolio testing to AB in the future (hopefully near future).
3a. One could write a short program in VB, C, or Delphi that would
wead out the stray trades before importing into Excel. There might
even be a way to write an Excel macro to do this (but I am not an
expert with Excel).
3b. There is another way that has recently come to mind (as yet
untried). First create a composite based on the market timing signal
(with "1" being given to signal change days and "0" to all other
days). Long entry could be in Open; long exit in High, short entry
in Low and Short cover in Close, or separate composites could be
used. Then using the CUM() function one could number each timing
signal and with an appropriate Optimize statement, one would end up
with trades that could be sorted in the Excel pivot table not by
date of entry but by the signal number (the optimization number).
Thus a stock that entered one day later than most would have the
same Optimization number as those that entered on the "right" day
and the Excel pivot tables would not get messed up by the stray date
entries.
Just writing this up makes me tried. I look forward to the day when
portfolio level testing gets added to AB.
b
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at:
http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|