RE: [amibroker] Re: How to save Metrics in Composites for Individual BTs, AmiBroker Email List Archive

Herman,

$0.02 ? … LOL … I think you got at least a nickels worth there …

My original reaction to Al’s statement / request was … “I agree that this would be nice to have as directly as you have laid out” …

… and while TJ’s example of how to do UPI in the CBI was about the simplest performance metric that one could have been done the rest are doable as well …

I only posted the simple OLE/Automation code because AL asked me to …

Personally I could care less if anyone else ever used OLE/Automation for anything but I probably have more things written with it then I do in AFL simply because it allows many things to be done that can’t otherwise be done although as I suggested and TJ has demonstrated that is not the case here.

I am in agreement that that Al’s request would have wide appeal and one would assume ?! that if it is available for a Portfolio backtest that it would also be available for an individual backtest not unlike what you were looking for the other day … Maybe someday that feature will be available from AB / AA / AFL as simply as Al has requested … Maybe that someday will be relatively soon … Maybe not …

I guess the questions you’ll need to answer for yourself are …

- How have you managed to survive this long without the feature Al is requesting ?

- How long are you prepared to survive without it into the future if that someday isn’t any time soon ?

- If you are not prepared to deal with it not being available for a long time, then what ?

Fred

From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf Of Herman
Sent: Monday, July 28, 2008 9:32 PM
To: amibroker@xxxxxxxxxxxxxxx
Subject: RE: [amibroker] Re: How to save Metrics in Composites for Individual BTs

Hello Fred,

Putting technical problems and limitations aside it is clear to me that having access to period-based performance metrics is critical in trading. Since the Equity curve is based on system performance it makes common sense that performance metrics will lead on the Equity. Most performance metrics can't be extracted from the equity curve; they are lost at that summary-stage. For example, DrawDown is the result of advanced system failure; it might very well have been possible to detect the upcoming failure (DD) by looking at some very basic metrics. Simply looking at the number of winning trades or profit/trade, might have alerted us. Using more sophisticated metrics, like a period based UPI Indicator, we might have seen a very clear and gradual system failure. All this is especialy true for faster trading systems where metrics become very smooth indicators.

To look at metrics for the entire duration of the backtest may come in handy during development however during real trading it is useless. It is simply due to a reluctance to break with tradition (a problem for most traders) that we don't have period-based performance indicators today. They are long overdue. Sure, they they are 'doable' in AmiBroker, but how many users would be able to write such code? In my opinion not more that 1%. To develop systems efficiently we need basic afl tools; no one wants to spend two months learning OLE only to find out that it doesn't fill the requirement or is to complicated for the user. Same for the CBT, who wants to spend two months learning it just to test and idea that may or may not work? Time is precious...we want to trade, not become professional programmers.

A few common applications for metric indicators are:

1) When trading a number of different systems one wants to know early when a system starts to fail. How else can one make a timely switch? Just like you may want to trade funds with a rotational system you want to rotate the trading systems themselves.

2) It is very common for trading systems to fade in and out of performance. To detect when a trading system fails and to switch to another system requires, again, monitoring system performance metrics. imo, There is just no other way around it.

3) And of course we want to know when a ticker loses its character and stops working. Here again performance metrics might be the best way to detect ticker failure.

A function like getPerformanceMetric ( MetricName, LookBackPeriod) would find wide application. Most people think right away of portfolio trading however performance metrics should be calculated for individual stocks, they will lose their meaning if derived from portfolio results.

I am not saying that it is easy to design such indicators, perhaps it is extremely difficult. But that doesn't remove the need for them.

Just my two cents worth :-)

Best regards,

herman

-----Original Message-----
From: amibroker@xxxxxxxxxps.com [mailto:amibroker@yahoogroups.com]On Behalf Of Fred Tonetti
Sent: July 28, 2008 12:44 PM
To: amibroker@xxxxxxxxxps.com
Subject: [SPAM]RE: [amibroker] Re: How to save Metrics in Composites for Individual BTs

Al,

What you are looking for in AB is I believe a little more difficult then or at least as time consuming as you think it is …

Share ? … Code ?

LOL … I thought I did in that post …

Ok … Kidding aside … Assuming that

-          AB is running

-          AA has the AFL you want to run ( This doesn’t mean the formula editor as the FE and AA aren’t coupled )

-          AA Settings for ApplyTo / Range etc are as you want then

This VBS code would run a BackTest and export the Trade List to a File

Dim oAB

Dim oAA

Set oAB = CreateObject("Broker.Application")

Set oAA = oAB.Analysis

oAA.Backtest(0)

oAA.Export("Dummy.csv")

The code above should be saved in a .vbs filetype ( name of your choosing ) and then simply double clicking it will produce the file.

This could easily be changed to running an optimize which will produce all the performance metrics in the AA results which could then be exported by changing …

oAA.Backtest(0)

to …

oAA.Optimize(0)

Ideally the optimize above would be a one step optimize if you will just to produce the performance metrics related to the backtest …

With a little more work i.e. a loop to set the beginning and ending dates one could get performance metrics externally for whatever lookback length one wanted one after the other …

From: amibroker@xxxxxxxxxps.com [mailto:amibroker@yahoogroups.com] On Behalf Of Al Venosa`
Sent: Monday, July 28, 2008 12:17 PM
To: amibroker@xxxxxxxxxps.com
Subject: Re: [amibroker] Re: How to save Metrics in Composites for Individual BTs

Dear TJ and Fred:

Thank you for these suggestions. However, what I am asking for is a simple AFL function in which we can specify the lookback period for the metric in question, nothing more. To observe the difference, you can simply substitute an MA() for the required function, as shown below. When you view the graph, you will see, I think, what I am talking about.

Plot(C,"",1,128);
Plot(MA(C,50),"FixedLookBack",colorRed,1); // the requested solution
Plot(MA(C,BarIndex()),"ExistingNow",colorBlue,1); // TJ's and Fred's solution.

I believe what TJ has suggested, if I am interpreting it correctly, is that I would be running a backtest for each bar in a loop, which would be complex and very slow. Maybe I'm asking the same thing. If so, tell me, and I'll desist. An alternative solution is to use the Walk Forward Individual Backtester to implement a system that uses performance metrics as position score.

BTW, Fred, are you willing to share the complete working code for your 2-line export?

Thank you.

Al V.

On 7/28/08, Tomasz Janeczko <groups@xxxxxxxxxxcom> wrote:

Hello,

It is doable with custom backtester and not so complicated.

As described in detail here:

http://www.amibroker.com/guide/a_custommetrics.html

You have direct access to ANY backtest performance metric using

GetPerformanceStats() function of backtester object.

There is no obstacle in calling it every bar and storing the result in the array if you wish.

Let say you want UPI as array.

// your trading system here
Buy = ...

Sell = ...

SetCustomBacktestProc("");

/* Now custom-backtest procedure follows */

if( Status("action") == actionPortfolio )
{
    UPI = 0;
    bo = GetBacktesterObject();

    bo.PreProcess();

    for( bar = 0; bar < BarCount; bar++ )
    {
      bo.ProcessTradeSignals( bar );

      st = bo.GetPerformanceStats(0); // get stats for all trades

      UPI[ bar ] = st.GetValue("UlcerPerformanceIndex");
   }

    bo.PostProcess();

    AddToComposite( UPI, "~~~UPI", "X", atcFlagDefaults | atcFlagEnableInPortfolio );
}

Now ~~~UPI ticker will contain bar-by-bar values of Ulcer Performance Index.

As for "specifying lookback period" it is doable by creating Xth composites (and X backtests) each containing values for specified

lookback period.

As for Equity() - this is SINGLE security (OLD) backtest. It has no comparision to portfolio level backtest that

must go through entire portfolio. The complexity of portfolio backtest is Nth times the single security backtest

where N is number of symbols in portfolio. Therefore it is not feasible to be calculated on-the-fly in real time like

single-security backtest (i.e. Equity()).

Best regards,
Tomasz Janeczko
amibroker.com

----- Original Message -----

From: Al Venosa`

To: amibroker@xxxxxxxxxps.com

Sent: Monday, July 28, 2008 1:02 AM

Subject: Re: [amibroker] Re: How to save Metrics in Composites for Individual BTs

Thanks, Fred, but I haven't a clue how to do OLE/Automation, and I wonder how many AB users out there really do. That's why I was calling for a simple, non-painful, easy-to-use AFL function that would do this for the non-techie/programmer, and if it existed, AB would be the only trading software out there that would be able to do this. I bet It would be a profit bonanza for TJ.

Al V.

On 7/27/08, Fred Tonetti <ftonetti@xxxxxxxxxxnet> wrote:

I agree that this would be nice to have as directly as you have laid out …

However, while somewhat painful, one can with Equity() and a list of trades calculate all the performance metrics as of any given bar or if you prefer as arrays of values …

This could be fully automated with OLE/Automation

From: amibroker@xxxxxxxxxps.com [mailto:amibroker@xxxxxxxxxps.com] On Behalf Of Al Venosa
Sent: Sunday, July 27, 2008 11:29 AM
To: amibroker@xxxxxxxxxps.com
Subject: [amibroker] Re: How to save Metrics in Composites for Individual BTs

Having read this interesting thread begun last week, I'd like to
continue it with a follow-on question/comment that I think, if TJ were
to implement it, would make Amibroker infinitely more useful to the
actual trader. It would be awesome if we could have simple-to-use AFL
functions that read AFL backtest metrics directly. Adding a lookback
period would make them immensely more useful as indicators. What I am
suggesting is to have a function like:

getEquityMetric ( MetricName,LookBackPeriod);

The example code provided by TJ gives us only the cumulative value of
each metric. What I'm suggesting is to go beyond this one cumulative
output number and create metric arrays with a specified lookback
period. Then, when we plot these metrics in an indicator, we can
visually look for correlations with price charts. For example, one
could plot winning trades/month and see if they change with trend. Or
one could look at AverageWin or UPI and see how that changes with
trend. These are all correlations that are best analyzed visually (in
an Indicator) but can ONLY be analyzed if we have access to these
metrics for variable lookback periods.

Another use for these functions would be as a positionscore in a
trading system. What better way is there to select tickers/systems to
trade than the actual performance of those tickers or systems? The
procedure may require a preliminary scan/exploration to create metric-
composites that can be read by the trading system and used as a
positionscore. Critical here is that the metric can be read for any
specified lookback period, i.e. 10 bars, 100 bars, etc. So the
function must have a period argument, which is the most important
factor. We already have equity(). Why not expand this with the other
backtest metrics?

Undoubtedly, all this can be implemented using the custom backtester,
but this solution probably excludes >95% of all AmiBroker users.

TJ, would this be possible to implement?

Al Venosa

--- In amibroker@xxxxxxxxxps.com, "Herman" <psytek@xxx> wrote:
>
> Thank you TJ :-) you saved the day once more !
>
> Great stuff.
>
> If someone is wondering why I wanted this program... You can design
trading
> systems and use performance metric arrays as powerful Indicators. It
is
> somewhat similar to trading the equity curve. Price arrays can have
> qualities that can make your trading systems fail but that are
undetectable
> with traditional indicators.
>
> However, you can design small trading systems that target specific
price
> characteristics, like patterns, trends, volatility, cycles, etc.
Using the
> code below gives you statistical information about these
characteristics in
> a form that can be plotted, and be used in other trading systems.
>
> Thanks everyone for your help!
> have a great trading day!
> herman
>
> // Demo trading system
> Short = Cover = 0;
> Buy = Cross( MACD(), Signal() );
> Sell = Cross( Signal(), MACD() );
> // Using the CBT to retrieve/save metrics
> if( Status("action") == actionBacktest ) StaticVarSetText( "Symbol",
> Name() );
> SetOption( "UseCustomBacktestProc", True );
> if ( Status( "action" ) == actionPortfolio )
> {
> bo = GetBacktesterObject();
> bo.PreProcess();
> MyHistStat1 = Null;
> for ( bar = 0; bar < BarCount; bar++ )
> {
> bo.ProcessTradeSignals( bar );
> stats = bo.GetPerformanceStats( 0 );
> MyHistStat1[ bar ] = stats.GetValue( "UlcerIndex" ); // any metric
can be
> retrieved
> }
> bo.PostProcess();
> AddToComposite( MyHistStat1, "~~~UI_" + StaticVarGetText
( "Symbol" ), "X",
> atcFlagEnableInPortfolio | atcFlagDefaults );
> }
> PlotForeign( "~~~UI_"+Name(), "UlcerIndex Historical", colorRed,
> styleLine );
>
> -----Original Message-----
> From: amibroker@xxxxxxxxxps.com [mailto:amibroker@xxxxxxxxxps.com]On
Behalf
> Of Tomasz Janeczko
> Sent: July 25, 2008 5:49 AM
> To: amibroker@xxxxxxxxxps.com
> Subject: [SPAM]Re: [SPAM]Re: [amibroker] How to save Metrics in
Composites
> for Individual BTs
>
>
> Herman,
>
> You forgot the CORRECTION I mentioned:
>
> StaticVarSetText( "Symbol", Name() ); must NOT be called
unconditionally,
> but THIS way:
>
> =====================================================================
> if( Status("action") == actionBacktest ) StaticVarSetText( "Symbol",
> Name() );
> ==============================================================
>
> Best regards,
> Tomasz Janeczko
> amibroker.com
> ----- Original Message -----
> From: Herman
> To: amibroker@xxxxxxxxxps.com
> Sent: Friday, July 25, 2008 11:36 AM
> Subject: RE: [SPAM]Re: [amibroker] How to save Metrics in Composites
for
> Individual BTs
>
>
> Still NO GO.
> I am loading the code in the AA, select a watchlist, run an
Individual
> backtest, and Refresh the WorkSpace. I get the BT report with
individual
> results. I get two Composites in my Composites Group. One is named
> ~~~EQUITY, the other ~~~UI_~~~EQUITY. The first makes sense but the
second
> indicates that the StaticVar does not return the ticker name.
>
> >> It appears that in this code the function Name() returns
"~~~EQUITY" and
> does not return the name for the ticker being tested, it behaves as
if the
> ~~~EQUITY composite is the ticker being tested.
> Can anyone confirm this?
>
> Thanks again!
> Herman
>
> // Demo trading system
> Short = Cover = 0;
> Buy = Cross( MACD(), Signal() );
> Sell = Cross( Signal(), MACD() );
> // Using the CBT to retrieve/save metrics
> StaticVarSetText( "Symbol", Name() );
> SetOption( "UseCustomBacktestProc", True );
> if ( Status( "action" ) == actionPortfolio )
> {
> bo = GetBacktesterObject();
> bo.PreProcess();
> MyHistStat1 = Null;
> for ( bar = 0; bar < BarCount; bar++ )
> {
> bo.ProcessTradeSignals( bar );
> stats = bo.GetPerformanceStats( 0 );
> MyHistStat1[ bar ] = stats.GetValue( "UlcerIndex" ); // any metric
can be
> retrieved
> }
> bo.PostProcess();
> AddToComposite( MyHistStat1, "~~~UI_" + StaticVarGetText
( "Symbol" ), "X",
> atcFlagEnableInPortfolio | atcFlagDefaults );
> }
> PlotForeign( "~~~UI_"+Name(), "UlcerIndex Historical", colorRed,
> styleLine );
>
>
>
>
> -----Original Message-----
> From: amibroker@xxxxxxxxxps.com [mailto:amibroker@xxxxxxxxxps.com]On
Behalf
> Of Tomasz Janeczko
> Sent: July 25, 2008 4:08 AM
> To: amibroker@xxxxxxxxxps.com
> Subject: [SPAM]Re: [amibroker] How to save Metrics in Composites for
> Individual BTs
>
>
> It will work OK.
> Individual backtest *is* portfolio backtest but just portfolio
consisting of
> one symbol at a time.
>
> Note that one should select "Individual Backtest" (not "OLD"
backtest) from
> AA->Backtest drop down.
>
> One correction though
> StaticVarSetText( "Symbol", Name() );
>
> should be called only when NOT in portfolio mode
>
> so
>
> if( Status("action") == actionBacktest ) StaticVarSetText( "Symbol",
> Name() );
>
> // Demo trading system
> Short = Cover = 0;
> Buy = Cross( MACD(), Signal() );
> Sell = Cross( Signal(), MACD() );
>
> // Using the CBT to retrieve/save metrics
> SetOption( "UseCustomBacktestProc", True );
> if ( Status( "action" ) == actionPortfolio )
> {
> bo = GetBacktesterObject();
> bo.PreProcess();
> MyHistStat1 = Null;
> for ( bar = 0; bar < BarCount; bar++ )
> {
> bo.ProcessTradeSignals( bar );
> stats = bo.GetPerformanceStats( 0 );
> MyHistStat1[ bar ] = stats.GetValue( "UlcerIndex" ); // any metric
can be
> retrieved
> }
> bo.PostProcess();
> AddToComposite( MyHistStat1, "~~~UI_" + StaticVarGetText( "Symbol" )
+
> "_HISTORICAL", "X", atcFlagEnableInPortfolio | atcFlagDefaults );
> }
>
> ----- Original Message -----
> From: Paul Ho
> To: amibroker@xxxxxxxxxps.com
> Sent: Friday, July 25, 2008 4:45 AM
> Subject: RE: [amibroker] How to save Metrics in Composites for
Individual
> BTs
>
>
> First of all. You use Status{"action") == actionPortfolio,
individual
> backtest wont go through there.
>
>
>
> From: amibroker@xxxxxxxxxps.com [mailto:amibroker@xxxxxxxxxps.com]
On Behalf
> Of Herman
> Sent: Friday, 25 July 2008 9:31 AM
> To: amibroker@xxxxxxxxxps.com
> Subject: Re: [amibroker] How to save Metrics in Composites for
Individual
> BTs
>
>
> Thank you Tomasz, but this code still does not work. I changed the
StaticVar
> to the Text type.
>
> Can you help some more ... ? or does anyone else see the problem?
>
> TIA,
> Herman
>
> StaticVarSetText( "Symbol", Name() );
> // Demo trading system
> Short = Cover = 0;
> Buy = Cross( MACD(), Signal() );
> Sell = Cross( Signal(), MACD() );
>
> // Using the CBT to retrieve/save metrics
> SetOption( "UseCustomBacktestProc", True );
> if ( Status( "action" ) == actionPortfolio )
> {
> bo = GetBacktesterObject();
> bo.PreProcess();
> MyHistStat1 = Null;
> for ( bar = 0; bar < BarCount; bar++ )
> {
> bo.ProcessTradeSignals( bar );
> stats = bo.GetPerformanceStats( 0 );
> MyHistStat1[ bar ] = stats.GetValue( "UlcerIndex" ); // any metric
can be
> retrieved
> }
> bo.PostProcess();
> AddToComposite( MyHistStat1, "~~~UI_" + StaticVarGetText( "Symbol" )
+
> "_HISTORICAL", "X", atcFlagEnableInPortfolio | atcFlagDefaults );
> }
> PlotForeign( "~~~UI_HISTORICAL", "UlcerIndex Historical", colorRed,
> styleLine );
>
> -----Original Message-----
> From: amibroker@xxxxxxxxxps.com [mailto:amibroker@xxxxxxxxxps.com]On
Behalf
> Of Tomasz Janeczko
> Sent: July 24, 2008 3:00 PM
> To: amibroker@xxxxxxxxxps.com
> Subject: [SPAM]Re: [amibroker] How to save Metrics in Composites for
> Individual BTs
>
>
> The same code. The only distinction is that you need to run
INDIVIDUAL
> backtest
> and use Static variable to save name
>
> StaticVarSet Text ("Symbol", Name() );
> // Demo trading system
> Short = Cover = 0;
> Buy=Cross( MACD(), Signal() );
> Sell=Cross( Signal(), MACD() );
>
> // Using the CBT to retrieve/save metrics
> SetOption("UseCustomBacktestProc", True );
> if( Status("action") == actionPortfolio )
> {
> bo = GetBacktesterObject();
> bo.PreProcess();
> MyHistStat1 = Null;
>
> for(bar=0; bar < BarCount; bar++)
> {
> bo.ProcessTradeSignals( bar );
> stats = bo.GetPerformanceStats( 0 );
> MyHistStat1[ bar ] = stats.GetValue("UlcerIndex"); // any metric
can be
> retrieved
> }
>
> bo.PostProcess();
> AddToComposite( MyHistStat1, "~~~UI_" + StaticVarGet Text
("Symbol") +
> "_HISTORICAL", "X", atcFlagEnableInPortfolio | atcFlagDefaults );
> }
>
> PlotForeign("~~~UI_HISTORICAL", "UlcerIndex Historical", colorRed,
> styleLine );
>

I am using the free version of SPAMfighter for private users.
It has removed 512 spam emails to date.
Paying users do not have this message in their emails.
Try SPAMfighter for free now!

I am using the free version of SPAMfighter for private users.
It has removed 512 spam emails to date.
Paying users do not have this message in their emails.
Try SPAMfighter for free now!

I am using the free version of SPAMfighter for private users.
It has removed 513 spam emails to date.
Paying users do not have this message in their emails.
Try SPAMfighter for free now!
__._,_.___

Please note that this group is for discussion between users only. To get support from AmiBroker please send an e-mail directly to SUPPORT {at} amibroker.com For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: http://www.amibroker.com/devlog/ For other support material please check also: http://www.amibroker.com/support.html

Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___