|
PureBytes Links
Trading Reference Links
|
--- In amibroker@xxxxxxxxxxxxxxx, "Barry Scarborough" <razzbarry@xxx> wrote:
>
> Currently I design systems with basic AFL, modify it to allow it to drive auto trading and then design it again for analysis using BarReplay. I am trying to design one system that can be used in all three environments.
That's what I suspected. And, that's why I wanted to voice caution against the direction you are heading. The backtester produces state. Any attempt by you to do the same will fail unless you are able to 100% replicate the AmiBroker logic. If your strategy is applied to a portfolio, as opposed to a single stock, the challenge will be very significant as your logic will need to manage the entire portfolio when producing your states for the current symbol (e.g. if last available dollar just got spent on symbol XYZ, or if you're already holding max allowed positions, or trade volume is too low for your position size, or you already hold a position, or... you'll have to skip the signal that you just found on symbol ABC).
> In the auto trading program I use states to keep track of what is going on and I use numPositions to verify the states since the trading program can cancel an order if it is not filled by the time the trend changes. That blows AFLs mind. I also use states to eliminate multiple signals due to indicator flopping around and causing multiple, unwanted trades during a bar. That blows AFLs mind too since ALF never sees this with static data.
>
> Also, my trading program does not perform anywhere near a back test of the system. A number of my programs back test over 20,000% / year based on trading about 50 times a day and a 1 minute chart. I read about IO and wanted to try to use the intelligent optimizer to develop a more robust system. I assume that since I have such a high back test result that the system is over optimized, even though I did not use optimize. When I run the formula with live data it barely makes money. It can gain $4000 one day and down that much the next. Another reason to use IO.
>
If you did not use Optimize, then your system is not over-optimized (unless the values you do use have come from someone else who DID optimize).
The fact that your backtest does not match your live trading, within some margin of error, suggests that you have not captured your live trade logic.
If you are unable to get an AmiBroker backtest to closely mimic your live trading, then IO will be of no help to you. After all is said and done, IO is still just an optimizer. If you are unable to capture your trade rules, IO will not be able to optimize them any more than AmiBroker could.
> So I was trying to figure out how to write my trading program so that it was compatible with auto analysis, set my program up in AFL so that I could define the states and keep track of the positions in an attempt to use AA and IO for analysis. The more I try the more incompatibilities I find in AFL. I guess AFL was designed as a charting and analysis program using static data, intraday or EOD and loses its mind when someone tries to analyze a live trading program.
>
I suspect that it is exactly the act of trying to define the state that is tripping you up. If you can express the rules as genuine rules, then AmiBroker will apply those rules and produce the correct state for you. That is the whole point of the backtester.
Yes, there will be differences between a backtest and what you actually ended up with. For example, AmiBroker will assume that an order got filled if the criteria was met, whereas in actuality the market may have moved too fast for you to have been filled on your order. The goal is to write all encompassing rules and trade sufficiently liquid instruments such that these false trades are not significant with respect to the backtest results, thereby allowing for optimization and analysis.
> I am not new at this. I have been coding AFL since 2003 and live trading programs for over a year. In all that time I have not tried to use AA because I knew it would not help with live trading programming. But I was intrigued with the ability of IO.
Yes, I've seen your posts over time.
However, it is my opinion that IO will be of no use to you until you can get AmiBroker to mimic your live trading. The key difference that IO brings to the table is its explicit in-sample sensitivity analysis during optimization (as opposed to the implicit sensitivy of CMA-ES as asserted by Tomasz). But, if you are unable to capture your rules, the sensitivity analysis will be applied on meaningless results.
> The code I appended to the last post works and keeps the states in an array. But for some inexplicable reason AFL will not back test and the chart will only shows the active state.
I'm bogged down with taxes at the moment. So, I haven't had a chance to look at your script any closer.
> That probably has something to do with using static variables to save the state from scan to scan. I really don't understand why AFL will not allow an array built from a static variables to be used by AA. It could be that AFL is building the arrays during every scan and only plots the values from the current state. That seems to be what is happening. Coming from C++ I don't understand why I can't put data in array cells and have it stay there and be used.
I believe (though have not tried) that you can access static arrays from within custom backtester. You just have to write the code to do so.
If you really, really, want to do all the state tracking yourself, then perhaps you should look at writing low level custom backtester code where *your* code explicitly enters and exits the trades based on your own logic. That would be my absolute last choice. But, AmiBroker will gladly produce all the output statistics for you.
Mike
> So that is where I am coming from and what I am trying to do.
>
> Thanks for the help,
> Barry
>
> --- In amibroker@xxxxxxxxxxxxxxx, "Mike" <sfclimbers@> wrote:
> >
> > Yeah, Flip is a tricky one to understand. But, the end result is that you have an array alternating between all 1's and all 0's
> >
> > e.g.
> > AmLong = Flip(Buy, Sell); might give the following:
> >
> > 1111111111000111110000000001111111
> >
> > indicating that you were long 10 days, then flat for 3, then long for 5, then...
> >
> > What is it that you are trying to do? If this exercise is for anything more than display purposes, then you may be begging for trouble. What you are asking for is to implement the AmiBroker backtester in a few lines of AFL. This has been discussed a few times in the forum. To my knowledge, it's never been done.
> >
> > One thing that did come out of a prior discussion was the following article by Tomasz. Perhaps you can modify it to iterate through the trades (either open trades bar by bar, or closed trades after all is completed), and construct a status composite for each symbol in which you have taken a position. I haven't tried this. But it may be do-able.
> >
> > http://www.amibroker.com/kb/2008/05/19/historical-portfolio-backtest-metrics/
> >
> > What that reiterates is that until you have actually run the backtester, you won't know for sure whether a position was/is/will be held.
> >
> > Mike
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "Barry Scarborough" <razzbarry@> wrote:
> > >
> > > I will have to digest what you are saying. I don't understand flip yet.
> > >
> > > While I was waiting for an answer to my post I tried something else. It keeps track of the states but will not back test and the shape arrows are not displayed correctly. If I am in long state the sell and short arrows are shown. If I am short the buy and cover arrows are shown. I have seen this before but I do not know why Ami does not back test and show the arrows correctly. Any ideas why?
> > >
> > > Thanks,
> > > Barry
> > >
> > > The code:
> > > r1 = Param( "Fast avg", 3, 2, 20, 1 );
> > > r2 = Param( "Slow avg", 9, 2, 50, 1 );
> > > r3 = Param( "Signal avg", 2, 2, 20, 1 );
> > >
> > > fMacd = MACD(r1, r2);
> > > fSig = Signal(r1,r2,r3);
> > > Up = fMacd > fSig;
> > > Dn = fMacd < fSig;
> > >
> > > pStoch = Param("Stoch period", 10, 1, 30, 1 );
> > > pKavg = Param("%K avg", 3, 1, 20, 1 );
> > > Line = Param("Line", 15, 5, 40, 1);
> > > fStoch = StochK(pStoch, pKavg);
> > > stoUp = fStoch > Ref(fStoch, -1);
> > > stoDn = fStoch < Ref(fStoch, -1);
> > >
> > > state = Nz(StaticVarGet("State"));
> > > Buy = (state == 0 OR state == 4) AND fMACD > fSig AND stoUp;
> > > state = IIf(Buy, 1, state); // set long state
> > > Sell = state == 1 AND fSig > fMACD;
> > > state = IIf(Sell, 2, state); // flat after long state
> > > Short = (state == 2 OR state == 0) AND fSig > fMACD AND stoDn;
> > > state = IIf(Short, 3, state); // set short state
> > > Cover = state == 3 AND fMACD > fSig;
> > > state = IIf(Cover, 4, state); // flat after short state
> > > printf("\nState1=" + NumToStr(state, 1) + " State2=" + NumToStr(Ref(state, -1), 1) + " State3=" + NumToStr(Ref(state, -2), 1));
> > > StaticVarSet("state", LastValue(state));
> > >
> > > printf("\nbuy=" + NumToStr(Buy, 1) + " sell=" + NumToStr(Sell, 1) + " short=" + NumToStr(Short, 1) + " Cover=" + NumToStr(Cover, 1));
> > >
> > > Plot(fMacd, StrFormat(_SECTION_NAME()+ "(%g,%g)", r1, r2), ParamColor("MACD color", colorRed ), ParamStyle("MACD style") );
> > > Plot(fSig, "Signal" + _PARAM_VALUES(), ParamColor("Signal color", colorBlue ), ParamStyle("Signal style") );
> > > Plot( fMacd - fSig, "MACD Histogram", ParamColor("Histogram color", colorBlack ), styleNoTitle | ParamStyle("Histogram style", styleHistogram | styleNoLabel, maskHistogram ) );
> > > Plot(0,"",colorBlack);
> > > Plot(state, "State", colorGreen, styleOwnScale) | styleStaircase;
> > >
> > > PlotShapes(Buy * shapeUpArrow, colorGreen, 0, fMACD, 5 );
> > > PlotShapes(Sell * shapeDownArrow, colorRed, 0, fMACD, 20 );
> > > PlotShapes(Short * shapeHollowDownArrow,colorRed, 0, fMACD, -5 );
> > > PlotShapes(Cover * shapeHollowUpArrow, colorGreen, 0, fMACD, -20 );
> > >
> > >
> > > --- In amibroker@xxxxxxxxxxxxxxx, "Mike" <sfclimbers@> wrote:
> > > >
> > > > A quick follow up to prevent (hopefully) any misinterpretation of what I was trying to say.
> > > >
> > > > 1. The ExRem is only there for the plotting. You generally shouldn't use it for the Buy/Sell/Short/Cover arrays since the backtester does it better.
> > > >
> > > > 2. I did not attempt to prevent a Short if already in a Buy or vice versa because you cannot know which takes precedence without looping bar by bar through the range under analysis (to see which would have occurred first). These are the decisions that the backtester makes with full information.
> > > >
> > > > Mike
> > > >
> > > > --- In amibroker@xxxxxxxxxxxxxxx, "Mike" <sfclimbers@> wrote:
> > > > >
> > > > >
> > > > > Barry,
> > > > >
> > > > > You cannot reliably do what you are attempting. The only way to truely
> > > > > know whether or not you are in a position is via iterating through the
> > > > > open position list within custom backtester code. The backtester employs
> > > > > numerous rules that affect whether or not a position is actually taken,
> > > > > despite the signals given (e.g. availability of funds, max positions,
> > > > > limits on number of positions per symbol, etc.).
> > > > >
> > > > > In answer to your other questions, BarsSince is entirely dependent upon
> > > > > the state of the arrays at the time that you call it. So, of course, it
> > > > > matters a great deal where you place the call in your code.
> > > > >
> > > > > To illustrate the problem, here is code that will do what you want. But,
> > > > > you will notice that it is wrong since it allows simultaneous positions
> > > > > of both long and short, does not take into consideration availability of
> > > > > funds, etc. all of which only the custom backtester would be able to
> > > > > filter reliably.
> > > > >
> > > > > MA1 = MA(Close, 5);
> > > > > MA2 = MA(Close, 25);
> > > > > MA3 = MA(Close, 7);
> > > > > MA4 = MA(Close, 35);
> > > > >
> > > > > Buy = Cross(MA1, MA2);
> > > > > Sell = Cross(MA2, MA1);
> > > > > AmLong = Flip(Buy, Sell);
> > > > >
> > > > > Short = Cross(MA4, MA3);
> > > > > Cover = Cross(MA3, MA4);
> > > > > AmShort = Flip(Short, Cover);
> > > > >
> > > > > State = IIF(AmLong, 1, IIF(AmShort, 2, 0));
> > > > >
> > > > > Buy = ExRem(Buy, Sell);
> > > > > Sell = ExRem(Sell, Buy);
> > > > > Short = ExRem(Short, Cover);
> > > > > Cover = ExRem(Cover, Short);
> > > > >
> > > > > Plot(Close, "Close", colorLightGrey, styleBar);
> > > > > Plot(State, "State", colorRed, styleStairCase | styleOwnScale);
> > > > > PlotShapes(Buy * shapeUpArrow, colorGreen);
> > > > > PlotShapes(Sell * shapeDownArrow, colorRed);
> > > > > PlotShapes(Short * shapeHollowDownArrow, colorRed);
> > > > > PlotShapes(Cover * shapeHollowUpArrow, colorGreen);
> > > > >
> > > > > Mike
> > > > >
> > > > >
> > > > > --- In amibroker@xxxxxxxxxxxxxxx, "Barry Scarborough" <razzbarry@>
> > > > > wrote:
> > > > > >
> > > > > > I am trying to set states based on whether I have a position and the
> > > > > position type. If AFL has a getPositions function I can't find it. I
> > > > > tried to create states using the following code but barssince does not
> > > > > return a valid value.
> > > > > >
> > > > > > // buy control
> > > > > > BarsSB = Nz(BarsSince(Buy), 0);
> > > > > > BarsSSl = Nz(BarsSince(Sell), 0);
> > > > > > BarsSSh = Nz(BarsSince(Short), 0);
> > > > > > BarsSC = Nz(BarsSince(Cover), 0);
> > > > > >
> > > > > > state = 0;
> > > > > > // 0 = no position, 1 = buy last, 2 = short last;
> > > > > > state = IIf(BarsSB == BarsSSh, 0, IIf(BarsSB < BarsSSh, 1, 2));
> > > > > > // if sell after buy we are flat
> > > > > > state = IIf(state == 1 AND BarsSSl < BarsSB, 0, state);
> > > > > > // if cover after short we are flat
> > > > > > state = IIf(state == 2 AND BarsSC < BarsSSh, 0, state);
> > > > > >
> > > > > > If I don't use NZ then BarsSince returns Empty.
> > > > > > When I use NZ then I can see trades using plot shapes. But all the
> > > > > barsSince returns 0 so state is always 0. What am I doing wrong?
> > > > > >
> > > > > > The really interesting thing is that I can put the barsince lines
> > > > > below the logic where I set buy, sell, short and cover, then I get
> > > > > correct values returned. Why does the position matter?
> > > > > >
> > > > > > I can use static vars to keep the state but when I do that
> > > > > AutoAnalysis will not work and the shapes don't show properly. So I am
> > > > > tryng to find a way that allows auto analysis to work. I tried to use
> > > > > switch and found it only works with numbers and not arrays.
> > > > > >
> > > > > > Thanks,
> > > > > > Barry
> > > > > >
> > > > >
> > > >
> > >
> >
>
------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|