|
PureBytes Links
Trading Reference Links
|
Hi, I am submitting the following code for comment. I believe that I
have a generic solution for the "fixed number of conditional orders
on next bar" problem.
Constructive criticism is welcomed. Please feel free to point out any
bugs, inefficiencies, suggested enhancements, etc.
If this is deemed valuable, I can clean up the formatting and add it
to the files section.
Mike
Problem Statement:
------------------
Need programatic support for simulating the placement of a fixed
number of prioritized *conditional* orders at the next bar, in
response to one or more favorable "setups" triggered by the current
bar, and constrained by the number of positions permitted by the
strategy.
Example:
--------
Consider a strategy that calls for the placement of limit orders at
the next bar after recognizing one or more favorable setups on the
current bar. Assume that the strategy allows a maximum of only 10
open positions at any given time, and that the setups are prioritized
from most preferred (position 1) to least preferred (position 10).
If on the first day of the strategy 20 favorable setups were
recognized, then the strategy would call for placing limit orders at
the next bar for *only* the top 10 preferred setups (since only 10
positions are permitted and we can not know in advance whether any or
all of the 10 orders would actually get filled).
Similarly, if at some point after starting the strategy we found
ourself with 2 currently open positions and received 20 setups, we
would place 8 (10 - 2 = 8) limit orders for the top 8 preferred
setups.
Complications
-------------
1. Using PositionScore and position sizing is not sufficient since
they do not recognize the allocation of funds commited to
*conditional* order placements that do *not* get filled. Resulting
code would typically continue to attempt to fill allowable position
count despite not having enough funds to cover all possible setups.
2. Script execution for any given symbol does not have access to the
PositionScore of the remaining symbols.
3. Custom backtester object does not have access to the PositionScore
of any symbol that did *not* result in a generated trade signal (i.e.
if a limit order was not met, the custom backtester would not have a
signal object for that conditional placement, and thus would not have
access to the PositionScore of the unsuccessful order).
Solution
--------
1. Generate a "composite" symbol for each allowable position of the
strategy (e.g. for a strategy allowing a maximum of 10 open
positions, geneare composites ~Position1, ~Position2, ...,
~Position10).
2. At each bar of each symbol, calculate a PositionScore for any
conditional order based on the recognition of a setup on the previous
bar (e.g. PositionScore for a limit order in response to a recognized
setup the bar before). Note that this is a PositionScore for the
*conditional* order which *may or may not* have been filled. If no
setup was recognized in the previous bar the PositionScore would be
zero.
3. Insert, in a sorted manner, the calculated PositionScore into the
appropriate composite, bumping down in a chain reaction any current
composite occupants as needed.
For example; if the PositionScore for the current symbol at the
current bar was found to be less than the value held by ~Position1
for that bar, the comparrison would next be made against ~Position2.
If the PositionScore was found to be greater than the value held by
~Position2 for that bar, then the value for that bar of ~Position2
would be replaced (bumped) by PositionScore, and the value that had
been in ~Position2 would be moved down to ~Position3 for that same
bar, bumping down any value held by ~Position3 in a chain reaction
until a zero composite value was found (i.e. nothing to bump) or all
composites had been updated.
e.g. given:
PositionScore[x] is 99 and;
~Position1[x] is 100
~Position2[x] is 50
~Position3[x] is 49
~Position4[x] is 0
...
~Position10[x] is 0
Result after insertion would be:
~Position1[x] is 100
~Position2[x] is 99
~Position3[x] is 50
~Position4[x] is 49
~Position5[x] is 0
...
~Position10[x] is 0
4. Write custom backtester logic to calculate, at each bar, how many
open positions exist, and reset to 0 the PosSize of any Signal whose
PositionScore is *less* than the PositionScore held by the N-th
composite, where N is calculated as max allowed positions minus the
number of currently opened positions (e.g. 10 max positions - 2 open
positions = composite ~Position8). This emulates not having placed
orders for any but the top N preferred setups.
5. Leave to the native backtester all other decisions regarding enty
into positions and tie-breaking of equal PositionScore.
Advantages
----------
1. Works generically for any conditional strategy, Long or Short*.
2. Works equally well for scale-in strategies.
3. Makes no assumptions regarding Buy/Sell rules.
4. Does not result in any phantom/artificial trades.
5. Does not generate any phantom/artificial commisions.
6. Does not depend on any backtester settings "tweaks".
7. PositionScore data is available at all times for additional
analysis.
* Backtester logic must be custom fit to your strategy, but
persistence of scores is generic to all.
Disadvantages
-------------
1. Slower execution resulting from heavy looping (loops N times more
than alternative proposed in msg #113384, where N equals maximum
allowed positions).
For example; A strategy backtested 3 months over 7800+ symbols using
10 allowed positions on a 1.4Ghz laptop with 1GB RAM takes about 10
minutes.
2. Code is more complex than alternative proposed in msg #113384.
-----
----- Sample code snippets for your review (Long only)
----- I have left in _TRACE statements to see what's happening
-----
/*
* Carry fixed number of positions, equally divided.
*/
maxPositions = 10;
PositionSize = -100/maxPositions;
SetOption("MaxOpenPositions", maxPositions);
/*
* Custom backtester implementation to strip out orders that in
* reality would not have been placed due to a limitation of
* available capital to cover bids on all setups.
*
* Note: This implementation assumes Long positions only!
*/
SetCustomBacktestProc("");
if (Status("action") == actionPortfolio) {
bo = GetBacktesterObject();
bo.PreProcess();
for (bar = 0; bar < BarCount; bar++) {
openCount = 0;
for (openPos = bo.GetFirstOpenPos();
openPos;
openPos = bo.GetNextOpenPos())
{
openCount++;
}
minPos = maxPositions - openCount;
posScores = IIF(minPos,
Foreign("~Position" + minPos, "X", 0),
9999); // Highest possible score!
for (sig = bo.GetFirstSignal(bar);
sig;
sig = bo.GetNextSignal(bar))
{
if (sig.IsEntry() AND sig.IsLong()) {
if (sig.PosScore < posScores[bar]) {
/*_TRACE(StrFormat("Score %9.4f less than top %1.0f scores
of %9.4f at bar %5.0f, cancel signal for ", sig.PosScore, minPos,
posScores[bar], bar) + sig.Symbol);*/
// Order would not have been placed, cancel it out.
sig.PosSize = 0;
}
}
}
bo.ProcessTradeSignals(bar);
}
bo.PostProcess();
}
/*
* For each bar following entry setup, persist PositionScore into an
* ordered list of Foreign symbols such that we may later have access
* to the top scores during backtesting, regardless of whether a
* concrete signal for the ranked symbol is actually found. See
* custom backtester method for filtering logic.
*
* For example; a 10 position, limit order strategy currently holding
* 2 positions might place limit orders for only the top 8 setups,
* despite recognizing more than 8 candidate setups. This method
* would sort the PositionScore of all candidate setups into 10
* foreign symbols, regardless of whether or not the limit order was
* met. The backtester would then compare the PositionScore of all
* filled signals (i.e. all signals of the limit price having been
* met), and cancel out those whose score was less than the top 8
* scores found in the Foreign symbols (i.e. cancel out signals for
* those symbols upon which, in reality, a limit order would never
* actually have been placed).
*
* Note: This implementation leaves the responsibility of tie-
* breaking to the backtester, when multiple symbols have the same
* PositionScore for a limited number of available positions. The
* symbol selected by the backtester may not be the one for which an
* order was actually placed in real life. But, the symbol selected
* is guaranteed to at least have an equivalent PositionScore. Use
* unique PosittionScore values, or trade in real life using the same
* tie-breaking logic that AmiBroker uses :)
*
* Note: This implementation assumes that PositionScore will be
* either zero, for bars not recognized as following order placement
* criteria (i.e. no setup from previous bar), or a non zero positive
* number, for bars where order placement criteria has been met. To
* reiterate, this refers to order *placement* criteria (i.e. setup),
* not order *fulfillment*.
*/
procedure persistScores(scores) {
local maxPositions; // Max positions allowed in portfolio
local empty; // Array of zeros
local bar; // Loop variable
local score; // PositionScore of bar-th bar
local pos; // Loop variable
local composite; // Name of pos-th composite
local posScores; // Scores persisted to composite
local delta; // Delta between PositionScore and composite
local affected; // Flag whether any deltas were added
maxPositions = GetOption("MaxOpenPositions");
empty = Cum(0);
for (pos = 1; pos <= maxPositions; pos++) {
/*_TRACE("Persist " + Name() + " to position " + pos);*/
composite = "~Position" + pos;
AddToComposite(0, composite, "X", 1 + 2 + 4 + 8); // Initialize
posScores = Foreign(composite, "X", 0); // Do not fill holes!
delta = empty;
affected = false;
for (bar = 0; bar < BarCount; bar++) {
if (scores[bar]) {
score = scores[bar];
if (score > posScores[bar]) {
/*_TRACE(StrFormat("Score %9.4f bumps down position %1.0f
score of %9.4f at bar %5.0f", score, pos, posScores[bar], bar));*/
// Grab current best value and hold for next composite
// iteratation, and calculate delta needed to add to
// this composite in order to make it equal new high score
scores[bar] = posScores[bar];
delta[bar] = score - posScores[bar];
affected = true;
}
/*else if (posScores[bar]) _TRACE(StrFormat("Score %9.4f
blocked by position %1.0f score of %9.4f at bar %5.0f", score, pos,
posScores[bar], bar));*/
}
}
if (affected) {
AddToComposite(delta, composite, "X", 1 + 2 + 4 + 8);
}
}
/*_TRACE("\n");*/
}
setup = ... // Some setup recognition logic
Buy = Ref(setup, -1) AND ... // Some conditional entry logic
PositionScore = IIF(Ref(setup, -1), ..., 0); // Some score logic or 0
Sell = ... // Some Sell logic
persistScores(PositionScore);
/*
* Example of 5% dip limit conditional entry:
* BuyPrice = min(Open, (Ref(Close, -1) * 0.95));
* Buy = Ref(setup, -1) AND Low <= BuyPrice;
*/
------
------ End code snippets
------
--- In amibroker@xxxxxxxxxxxxxxx, "ed2000nl" <empottasch@xxx> wrote:
>
> i'll try to reply directly from yahoo. My posts aren't coming
through
> anymore. Might be because of the ISP ...
>
>
> so if I understand correctly the problem is not that the backtester
> picks 20 stocks while there is only money for 10 but you have a
problem
> with the fact that if the 10 actual signals are not filled it will
use
> the lower ranking 10 signals and this is not what you want. You
can
> include this in the backtester. I explained the same thing some
time
> ago. For signals that you actually want to enter but in real life
will
> not be entered because the limit is not reached then you can tell
the
> backtester to enter at the open and exit at the open and do not
allow
> for a single bar trade (in the settings window). There might be
> easier ways to do this (I mean using arrays only) but I have some
> example code below. I did not check if the code is entirely correct
but
> I'll explain the idea: The Buy array is fed to the sellAtLimit
procedure
> and when it finds a buy it will check if the buy limit is reached
for
> that signal. If it is not reached (so if Low[ i ] >= buyLimit[
i ] )
> then you tell the backtester to enter and exit at the same price,
same
> bar. What happens is that the backtester reserves this money for
this
> trade and will not use it for another trade. The only thing that
is not
> realistic is that you will pay commission. But this will be a small
> factor. rgds, Ed procedure
> sellAtLimit_proc(Buy,BuyPrice,buyLimit,sellLimit) {
>
> global Sell;
> global SellPrice;
> global BuyAdjusted;
> global BuyPriceAdjusted;
>
>
> // initialise arrays
> SellPrice = 0;
> Sell = 0;
> BuyAdjusted = 0;
> BuyPriceAdjusted = 0;
>
> for (i = 1; i < BarCount; i++) {
>
>
> // case where it is likely to enter a long position
> if (Buy[ i ] == 1 AND Low[ i ] < buyLimit[ i ]) {
>
>
> // buy at limit
> BuyAdjusted[ i ] = 1;
>
> if (Open[ i ] < buyLimit[ i ]) {
>
>
> BuyPriceAdjusted[ i ] = Open[ i ];
>
> } else {
>
> BuyPriceAdjusted[ i ] = buyLimit[ i ];
>
> }
>
>
> // find a sell position + sellprice
> for (j = i; j < BarCount; j++) {
>
> if (O[ j ] > sellLimit[ j ]) {
>
> Sell[ j ] = 1;
> SellPrice[ j ] = O[ j ];
> i = j;
> break;
>
> } else if (O[ j ] < sellLimit[ j ] AND H[ j ] > sellLimit
[ j
> ]) {
>
> Sell[ j ] = 1;
> SellPrice[ j ] = sellLimit[ j ];
> i = j;
> break;
>
> } else if (j == BarCount - 1) {
>
> i = BarCount;
>
> }
>
>
>
>
>
> }
>
> } else if (Buy[ i ] == 1 AND Low[ i ] >= buyLimit[ i ]) {
>
> // enter and exit at the same price and time ("VOID" trade)
> BuyAdjusted[ i ] = 1;
> BuyPriceAdjusted[ i ] = Open[ i ];
>
> Sell[ i ] = 1;
> SellPrice[ i ] = Open[ i ];
>
>
> }
>
> }
>
>
> } // end procedure
>
>
> --- In amibroker@xxxxxxxxxxxxxxx, "sfclimbers" <sfclimbers@> wrote:
> >
> > Thanks for your reply. I will look into your suggestion, but I
don't
> > think that that is the issue that I am up against. I have actual
> > trade data from months of live trading. I am now trying to
backtest
> > the strategy used, and match the results to the actual data.
> >
> > My script, as written, is correctly entering and exiting with the
> > correct number of shares and correct price points on all the
correct
> > days for all the trades that actually took place.
> >
> > The problem is that if I receive 20 "go long" signals on Monday
> > night, but only have enough money to afford 8 more positions,
then in
> > real life I only place limit orders for the *top* 8 of the 20
> > candidates, not all 20.
> >
> > This means that in reality, if none of the top 8 dip to my limit
> > order, then I will not get any fills on Tuesday, even though I
still
> > have not filled my slots, and even though some of the lesser
> > candidates would have resulted in a fill had I place an order for
> > them.
> >
> > However, the script is considering *all* 20 candidates, and fills
up
> > to 8 that dip enough to trigger a limit order. In other words, the
> > script assumes that there are limit orders on all candidates
instead
> > of only the top 8.
> >
> > Using position score and position sizing is not enough, since
these
> > assume that the universe of candidates fitting the criteria is
always
> > available for prioritizing and filling available slots. But, in
> > reality, only a subset are being bid on.
> >
> > As an example, if I'm currently holding:
> > AAA, BBB
> >
> > And I then get signals for (in sorted order):
> > CCC, DDD, EEE, FFF, GGG, HHH, III, JJJ, KKK, LLL, ... TTT
> >
> > I will only place limit orders for the top 8:
> > CCC, DDD, EEE, FFF, GGG, HHH, III, JJJ
> >
> > If none of the top 8 above reach my limit, but say 8 lesser ones
do
> > (that I did not bid on), then in real life I will get no fills for
> > the day. However, my script is saying that I picked up the 8
lesser
> > fills since I had 8 slots open and these 8 met the limit price.
> >
> > How can I structure my code to recognize that 20 entry setups were
> > found, but only 8 of them were acted upon, none of which actually
> > worked out due to not meeting the limit price?
> >
> > I can't seem to use the custom backtester to sweep through the
orders
> > and null out the false buys that would not have taken place,
since I
> > don't have access to the scores of the candidates that didn't get
> > filled.
> >
> > Yet, similarly, I can't seem to prevent triggering the buys in the
> > first place, since I don't have access to the scores of the other
> > candidates at that time either.
> >
> > When there are fewer signals than slots to fill, everything is
> > great :) But this strategy often results in more signals than
there
> > is money to bid with :(
> >
> > Thanks.
> >
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "Edward Pottasch" empottasch@
> > wrote:
> > >
> > > hi,
> > >
> > > the way you set it up it shoudl not be possible. However, what
can
> > happen is that the backtester finds exits for the next day and
> > immediatelly fills them with new positions. So you need to make
sure
> > that you first exit your positions and tell the backtester to
enter
> > only on the next bar. This is usually the problem. There are
several
> > ways to achieve this. Maybe you will get a more satisfactory
result
> > when you set settradedelays(1,1,1,1).
> > >
> > > I use setttradedelays(0,0,0,0) but I make sure that the trade is
> > entered 1 bar after the signal (same with the exits),
> > >
> > > Ed
> > >
> > >
> > >
> > >
> > > ----- Original Message -----
> > > From: Michael White
> > > To: amibroker@xxxxxxxxxxxxxxx
> > > Sent: Friday, August 24, 2007 11:37 AM
> > > Subject: [amibroker] How do I backtest placing a restricted
> > number of limit orders each night?
> > >
> > >
> > > Can anyone help me model the following scenario?
> > >
> > > - Assume a portfolio is allowed to consist of some fixed
number
> > > of "slots" with equity equally divided among them (e.g. 10
slots
> > at
> > > 10% of equity).
> > > - Check for setup criteria at close of each day.
> > > - Place next day limit buy orders for as many unfilled slots
as
> > are
> > > currently available (e.g. if already have 2 fills after day 1,
> > then
> > > there are only 10 - 2 = 8 slots remaining for day 2, etc.).
> > > - Buy orders are prioritized by a calculated value.
> > >
> > > My problem is that if I receive a setup for more symbols than
I
> > have
> > > available slots (e.g. receive 20 setups but only have 8
available
> > > slots), my script will try to fill all 8 slots from the 20
> > > candidates, and the portfolio manager will correctly prevent
me
> > from
> > > having more positions than allowed (e.g. no more than 10).
> > >
> > > However, in reality, I will only have placed as many limit
orders
> > as
> > > I have available slots (e.g. 8 limit orders when 8 available
> > slots,
> > > not limit orders for all 20 candidates, since I only have
funds
> > to
> > > cover placing 8 orders).
> > >
> > > What is happening is that my script is filling orders that I
> > would
> > > not have placed! I need a way to indicate that despite 20
setups,
> > > only 8 limit orders were placed.
> > >
> > > Following is some script snippets.
> > >
> > > /*
> > > * Assume an initial purse and brokerage fees ($0.01/share)
> > > */
> > > SetOption("InitialEquity", 50000);
> > > SetOption("CommissionMode", 3);
> > > SetOption("CommissionAmount", 0.01);
> > >
> > > /*
> > > * Carry fixed number of positions, dividing 100% of Equity
between
> > > * them (based on previous bar's closing).
> > > */
> > > PositionSize = -100/10; // Each position is 10% of equity
> > >
> > > SetOption("MaxOpenPositions", 10); // No more than 10
positions
> > > SetOption("UsePrevBarEquityForPosSizing", True);
> > >
> > > /*
> > > * We recognize the sale signal at the close of a bar and
execute
> > the
> > > * sale at the open of the next one, delay sale by 1 day.
> > > */
> > > SetTradeDelays(0, 1, 0, 0);
> > >
> > > /*
> > > * Trigger a Buy signal when previous bar meets the setup
> > > * requirements AND this bar's Low has dropped to less than a
fixed
> > > * percentage below the previous bar's close. This emulates
having
> > > * placed a limit order the night before after having seen the
> > signal
> > > * on that day's close.
> > > */
> > > setup = ... // Some position entry logic.
> > > PositionScore = ... // Some prioritization logic.
> > >
> > > BuyPrice = Ref(Close, -1) * 0.95;
> > > Buy = Ref(setup, -1) AND Low <= BuyPrice; // Problem here!!!
> > >
> > > Sell = ... // Some sell logic.
> > >
> > > As indicated in my earlier comments. The problem is that in
> > reality I
> > > will not actually have placed orders for all candidates, but
> > rather
> > > only for as many as there are available slots (e.g. 8).
However,
> > the
> > > script will attempt to fill the available slots based on all
> > > candidates (e.g. 20).
> > >
> > > How can I restrict the Buy assignment to only apply to the
top X
> > of Y
> > > candidates based on priority (e.g. top 8 of 20 in example
above).
> > >
> > > Thanks in advance.
> > >
> >
>
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|