|
PureBytes Links
Trading Reference Links
|
Ken,
You are right. I avoided the more complex task of testing a
portfolio that does not use market timing.
My interest is primarily in the area of market timing systems. In
part, this is because one can prototype such systems using various
indexes. Testing one index at a time gets around the problems of
varying numbers of stocks being open at any given time. One can use
AB's super built in equity curve and just click on the variuos
indexes to get a good idea of how profitable and consistent a timing
system is. AB is just super for this prototyping.
Your post takes us into a more complex area. How does one evaluate
systems one plans to use to trade 10 open positions when the entry
and exit signals are based on the stocks themselves and not a market
timing signal?
When AB has portfolio level testing, this should be easy to handle.
You would put some sort criteria into you code that AB would use to
create a ranked array of tickers for each day. AB would then fill
the portfolio with your set number of stocks (say 10) starting at
the top of the ranked list for the day. Whenever a stock is exited,
AB would just add a new one from the ranked list for that day.
But until AB gets such an ability, what can we do to estimate the
effectiveness of such a system? You could try using the a pair of
optimization statements to segment the results by year and month.
OKyear = YEAR() == Optimize("TESTyr",2000, 1985, 2003, 1);
OKmonth = MONTH() == Optimize("Testmon", 1, 1, 12, 1);
BUY = OKyear AND OKmonth AND ...your code...
SELL = ...your code... // do not use OKyear and OKmonth to exit.
Some months will have a lot more trades than other months, but this
should give a rough idea of have consistent your system is. This
will measure consistency, but do not attempt to compound monthly
results because some trades might last for 2 or 3 months. In other
words, the month by month seqmentation of results can not be used to
estimately annual returns.
You also mention the very practical issue for actual trading of how
to decide which of 3 of 40 possible stocks to enter (after you have
exited 3 stocks). Persumably you would rank the 40 in some way even
if just informally. Can you identify what explicit rule you might
use to approximate how you would make your top picks? If you can,
then you can use what I call the "bin" method with AB's optimization
function to approximate sorting results.
Here is sample code for creating a simple bin segmentation test. For
illustrative purposes, I will use a volume surge as the sort. A more
useful sort might be a combined price and volume surge but it would
be more complex to set appropriate bin sizes and level (mainly by
trial and error).
SortKey = V/MA(V,30) ;
BinSize = 1;
BinBottom = optimize("BinB", 3, 2, 5, BinSize);
BinTop = BinBottom + BinSize;
InBin = SortKey >= BinBottom AND SortKey < BinTop;
BUY = InBin AND OKyear AND OKmonth AND ... your code...;
Hope this helps and does not confuse.
Now lets get a bit more complex. What happens if I am testing a
system like the one you describe, but in addition to the regular
exits based upon the action of the open trades, I also want to exit
a stock if I see a better trade. That happens in real life, but how
could that be back tested. I am sure there is a way to do it
although it might require one to write the code in AB, C, or Delphi.
Here is how its logic could go:
For each data day, create a ranked list of the top 50 stocks
according to the user's preferred sort.
Open trades using the top 10 stocks from the sort array.
Exit any trade that hits its profit stop, loss stop, trailing stop,
or "time" stop. Replace with the highest ranked stock on the list
for that day provided it is not already an open trade.
Now for the new part: Exit any stock that is no longer in the top 10
of the sort list. Replace with whichever stock is new to the top 10
on the sort list.
b
--- In amibroker@xxxxxxxxxxxxxxx, "Ken Close" <closeks@xxxx> wrote:
> B: what a great response with lots of things to think about.
>
> About all I can say at this moment, without further reading and
careful
> study of all that you have written, is that I do not think I saw
you
> refer to the following situation:
>
> Say you have a system but it generates different entry and exit
signals
> for individual stocks. There is not one market timing signal with
the
> associated task of managing many stocks all entering or exiting to
the
> tune of the market signal.
>
> Rather, each stock generates its own signals and you enter and
exit as
> you wish.
>
> If you have two stocks in your basket, it is easy to inspect and
take
> the trades. If you have 50 or 75 stocks, any number of which can
have
> signals on any day, then you have a much messier portfolio
management
> problem.
>
> Do you take 5 out of 15 signals on a date? And which of the 15 do
you
> take?
>
> Once you take 5 what happens the next day when there are new
signals on
> different stocks? OK, you specify a "hold 10 max" stocks.
Therefore,
> on the second day you could take 5 additional trades (you already
hold
> 5). Which new 5?
>
> Next day, more signals...now you can not take any of these until
one of
> these 10 that you hold then sell and get removed from the
portfolio.
> Say 2 are sold, so now you have room to take 2 more. And so it
goes.
>
> I have played with the PivotTable and I do not think arranging by
dates
> is enough to manage the above situation...at least I have not
found the
> way.
>
> Will Tomasz' future work take this into account? Perhaps so,
perhaps
> not?
>
> And finally what do you do today (assuming you have such a system
> delivering nice juicy stock picks each day)? Well, you manage it
by
> intuition with little chance of having any kind of backtested
results.
>
> At least I think that is the case.
>
> Thanks for taking the time to write your thoughts. I hope other
join in
> this discussion to broaden my horizons as to what might be
possible.
>
> Ken
>
> -----Original Message-----
> From: b519b [mailto:b519b@x...]
> Sent: Thursday, March 13, 2003 11:22 PM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: [amibroker] Work Arounds until AB gets Portfolio Testing
>
> --- In amibroker@xxxxxxxxxxxxxxx, "Ken Close" <closeks@xxxx> wrote:
> > Please share your workaround and other thoughts
> > and ideas you have about how to evaluate multiple
> > simultaneous positions.
>
> Ken, Steve, and Chuck:
>
> My workarounds involve varying degrees of extra "work". I have not
> put a lot of time into refining such workaround because they will
> all be obsolete when Tomasz adds portfolio testing to AB.
>
> First some background. In its present state, AB does a great job
> backtesting and optimizing SINGLE stocks. However, when
backtesting
> and optimizing groups of stocks various "issues" arise. There may
be
> additional issues, but I am aware of the following:
>
> A - The bais problem arises because there will be more trades in
> soee time periods than others -- sometimes a lot more. When one
time
> period gives a lot more trades than another time period, the
summary
> results will be biased to the period with the most trades.
>
> B - One implication of A, is that earlier years will in general be
> given less weight in the final results than more recent years.
This
> is for 3 reasons. First there are more stocks being traded today.
> Second, databases with survivorship bias will give further weight
to
> more recent years. Third, minimum average volume filters will
> exclude a greater percentage of stocks 12 years ago than 1 or 2
> years ago (trading volume has increased).
>
> C - Another implication is a bit harder to explain. Let's say one
is
> testing a strategy that will pick 10 or 15 stocks in an average
year
> and these picks tend to loose money, but once every 3 or 4 years
> market conditions are such that the strategy picks 100 stocks all
at
> once and on this and only this time the strategy produes big
> winners. The summary back test results will suggest you have a
real
> winner when you really are looking at a strategy that can be
> expected to loose most of the time.
>
> D - The opposite of C is also possible. A strategy that may only
> pick a dozen stocks most times and these are over all winners may
be
> rejected if only time in the test range, the strategy picks 150
> stocks but this is the one time the strategy gives a significant
> drawdown.
>
> E - If one attempts to use the AddToComposite function, special
care
> is needed so it will not make a loosing system appear to be a
winner
> just because new stocks are added to the Composite each year.
>
> F - Optimize a market timing method with a basket of stocks is, in
> my opinion, very prone to giving curve fit results. The reason is
> that the optimization just has to find 1 or 2 times periods that
> generates a lot of winning trades which can mask the fact the
system
> generates losses most of the time. Extra diligence is needed when
> evaluating optimization results. Some of the usual methods for
> detecting curve fitting will not work (such as scanning a summary
> list of optimization results to see if there are loss canyons --
the
> loss canyons might be there but be masked by dominance of a
mulitude
> of winning trades for one signal period).
>
> OK, enough back ground on the underlying issue (a varying number
of
> trades for each signal) and possible problems that come from it.
> What are some work arounds? There is a quick workaround and a slow
> one. The slow one is best but it is -- slow, so I use the quick
one
> more often.
>
> 1. A Quick Work Around. The purpose of this work around is to
reduce
> the possibility of a large group of trades in a single time
> dominating the results. I use the Optimization feature to group
> results into years. For a long system, I just add a statement like
> this:
>
> TESTyear = Optimize("TESTyear", 2000, 1985, 2003, 1);
> OKyear = TESTyear == YEAR();
> BUY = OKyear AND ...your buy code...
>
> 1a. Note: If you use Exit and Reverse systems, you can not use
> COVER = BUY; //because some trades will never exit if the trades
> normal exit signal comes in the next calendar year.
>
> 1b. What one wants to see is a system that is profitable for every
> year in the optimization results list.
>
> 1c. One could, although I have not, make shorter periods than 1
year
> by using a second optimization line using MONTHS.
>
> 2. The slower but best work around is to have AB do a back test
and
> then export the entire trade list to Excel. Use Excel's Pivot
Table
> feature to group the trades by entry date of the trade. This
assumes
> the system being tested is based on a market timing signal so that
> all the trades in one batch will have the same entry date. In the
> data section of the table, set the results to average (rather than
> sum which is the default I think).
>
> 2a. I prefer to put AB's "percent profit" figure into the data
> section of Excel's pivot table, but you might prefer to focus
> another item. Use "profit" with care since the actual amount will
> vary from stock to stock in the later periods due to compounding
> (assuming the system is profitable).
>
> 2b. One major headache is caused by "stray" stocks that enter a
> trade one day behind most of the other stocks (again I am talking
> about systems that have entries based on market timing signals). I
> expect this one day late entry is due to how AB fills data wholes
in
> individual stocks since it even occurs when I use the CROSS
function
> to make sure there is only 1 entry date for each market timing
> cycle. Whatever the reason, even just a few "stray" trade entries
> really mess up the Excel pivot table. Each of the stray date
stocks
> will get its own row on the pivot table and thus a single stray
will
> appear to have equal weight to 20 or 50 stocks that all have the
> same entry date.
>
> 2c. To get detect these strays, I add a new column to Excel before
> making the pivot table. This column compares the entry date of a
> stock with that of the previous stock and puts a flag in if they
> differ. Then I scan the column and delete any stocks that are
> strays. As you can imagine, this is time consuming, so I only do
it
> for the most interesting strategies.
>
> 3. Other ideas I have considered but not yet attempted. The reason
I
> have pursued these is my expectation that Tomasz will be adding
> portfolio testing to AB in the future (hopefully near future).
>
> 3a. One could write a short program in VB, C, or Delphi that would
> wead out the stray trades before importing into Excel. There might
> even be a way to write an Excel macro to do this (but I am not an
> expert with Excel).
>
> 3b. There is another way that has recently come to mind (as yet
> untried). First create a composite based on the market timing
signal
> (with "1" being given to signal change days and "0" to all other
> days). Long entry could be in Open; long exit in High, short entry
> in Low and Short cover in Close, or separate composites could be
> used. Then using the CUM() function one could number each timing
> signal and with an appropriate Optimize statement, one would end
up
> with trades that could be sorted in the Excel pivot table not by
> date of entry but by the signal number (the optimization number).
> Thus a stock that entered one day later than most would have the
> same Optimization number as those that entered on the "right" day
> and the Excel pivot tables would not get messed up by the stray
date
> entries.
>
> Just writing this up makes me tried. I look forward to the day
when
> portfolio level testing gets added to AB.
>
> b
>
>
>
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
>
> Your use of Yahoo! Groups is subject to
> http://docs.yahoo.com/info/terms/
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|