Re: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?], AmiBroker Email List Archive

On May 18, 2008, at 6:22 PM, Fred Tonetti wrote:

Dennis,

I responded to what you posted as opposed to what you may be doing … Then again I’m not at all sure how I was supposed to know specifically what you were doing beyond what you chose to post.

No problem. I was not trying to make any sort of point, just showing a simplified general flow of my work since you had ask in that direction.

Why not post the AFL since there shouldn’t be anything secretive about analyzing an equity curve and/or the corresponding trade list. I’m sure there are many who would get something out of the routines to calculate a variety of performance metrics from an equity curve and trade list.

LOL. I am having enough problems trying to find the time to finish my Flexible Parameters UKB article (and I am not even sure that anybody really cares if I do). Then I have chart button stuff I want to write up next. I have a number of generic pieces to my trading platform that I am willing to share as I have time and confidence that I have put it together in a useful and well thought out way for others. When I get to the point of not rewriting an area for while, then I think it is stable enough to share. I feel responsible for what I share, so I don't want it to be a headache for myself or anyone else. Besides, as I said, I am working on this section right now. I have only implemented some basic metrics so far, and am still thinking about and writing additional ones I want to use.

If you choose to post something the rest of us can get our teeth into so that the rest of us can understand the value of it fine … If not then don’t be surprised if generalities where one can not see any benefit using something other then the backtester that already exists are summarily dismissed.

No problem here either. Even though I would like to think so, I don't claim what is good for me is good for anyone else. In fact I am such a rebel in my thinking, I would be surprised if anyone else who is capable of using my stuff would even want to, because they would be such rebels that they would be doing their own thing. ;-)

Prior to having portfolio trading capabilities in AA, I wrote a very long AFL to do exactly that ( Portfolio Trader ) … However, it is ancient history and for the most part no longer needed which btw is fine with me as I’d really rather not have to do the bean counting routines that a backtester by definition must do.

Writing code to either have AA do what I want it to do and/or interpret the results is something else again.

Fred

If I were doing portfolio trading, I doubt that I would write my own BT --just too hard. I wrote one about 10 years ago just for EOD trading.

BR,

Dennis

From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf Of Dennis Brown
Sent: Sunday, May 18, 2008 6:07 PM
To: amibroker@xxxxxxxxxxxxxxx
Subject: Re: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?]

Fred,

Very nice post.

Thank you.

What I do is in a different vein than the built-in BT/CBT is designed for --I did not give all the flow for MM and multiple combined simultaneous systems that I am working on. How I do it (the hard way) actually comes out simpler for my specific needs. Others most likely have very different needs from mine.

BR,
Dennis

On May 18, 2008, at 5:33 PM, Fred Tonetti wrote:

Lots of work for no particular reason … See my previous post

From: amibroker@xxxxxxxxxps.com [mailto:amibroker@xxxxxxxxxps.com] On Behalf Of Dennis Brown
Sent: Sunday, May 18, 2008 5:24 PM
To: amibroker@xxxxxxxxxps.com
Subject: Re: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?]

Fred,

I constructed my back tester in stages. First I generate "signals" with indicators and assign them to categories like entries, exits, or trends. Then I combine the signals (including things like converting conditional executions like limit orders or stops) into actual buy/sell, short/cover arrays and prices. Finally, once I have all the execution arrays, I run them through a stats loop that generates all the stats numbers and arrays, which is actually the simplest loop of all.

I agree with your last statement. If it is possible to do it, it can be made easy to do it. Chart buttons or ParamTrigger() could also be a way to run it.

BR,
Dennis

On May 18, 2008, at 4:42 PM, Fred Tonetti wrote:

How do you get at the data needed for the Trade oriented metrics as opposed to ones that are only equity oriented ?

I suppose even this is possible via the buy and sell arrays if they are properly constructed, but this is hardly what I’d call a universal solution.

For the other 99+% I’m not sure what the big deal is in running a script which can be simply incorporated into the tools dropdown to get what one wants.

From: amibroker@xxxxxxxxxps.com [mailto:amibroker@xxxxxxxxxps.com] On Behalf Of Dennis Brown
Sent: Sunday, May 18, 2008 4:07 PM
To: amibroker@xxxxxxxxxps.com
Subject: Re: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?]

Fred,

I only trade one type of future contract at a time, so it is "easy" to write any performance metric I want, including an array of a normally single value type metric over time. I am a simple person, so I like to keep things simple so I can understand them.

BR,
Dennis

On May 18, 2008, at 3:51 PM, Fred Tonetti wrote:

… and you get performance metrics from an indicator how ? …

From: amibroker@xxxxxxxxxps.com [mailto:amibroker@xxxxxxxxxps.com] On Behalf Of Dennis Brown
Sent: Sunday, May 18, 2008 3:45 PM
To: amibroker@xxxxxxxxxps.com
Subject: Re: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?]

Please pardon my jumping in here with a comment. I don't really belong in this thread, because I wrote everything I need for backtesting my systems in AFL only (yes, in indicator mode), so I can of course write whatever I need in AFL.

However, it seems to me that there are just two basic philosophical concepts that need to be clarified.

1. Is it reasonably possible to do what you want with the tools provided?

2. Is it easy to do what you want with the tools provided?

I contend that if it is possible, then it can be made easy without changing the tools. The missing ingredient is simply someone who understands how to do it, leading the way by creating the example for others to follow.

I may not understand if something is possible, hard, or easy when I look at what I want to accomplish. After looking around for examples, I may conclude that it is one of those three cases. If it is easy, it is because I found an example that is close to what I want. If it is hard, it is because I found an example that is far from what I want, but gives me some idea about out to approach the problem. If I can find no examples at all, I am left with a question if it is even possible and possibly a waste of my time trying to do it. That is when I ask for advice here about the possibility, hoping for the unfortunately rare definitive answer one way or the other.

If something that is not currently possible is needed to make money with the system (and yes, sometimes UI things that might be considered just aesthetics to an EOD person do fall into that category for day traders who have only one second to take an action or lose money), It should be addressed by TJ in AB with some priority.

If it something that is possible, but not easy, then it needs to be addressed by creating examples of how to do it, or even creating #include samples that make it as easy to deal with as if it were built-in. This can be done by TJ, or by other advanced users and made available in some convenient place like the UKB.

It is important to understand which category a problem falls into so that the appropriate remedy can be pursued. We will get the most out of AB by being clear about this and making suggestions for additions to AB/AFL where no good alternative exists, and asking for good, well explained examples where the solution exists, but is not obvious.

It seems that this thread started with a lack of clarity of what exactly is needed first, and how to get the desired result from AB second. The result is several frustrated people not connecting with a common understanding of the problem much less a solution.

(Yes, I get frustrated also, when I don't seem to be able to communicate my needs in a way that results in good solutions.)

If you are still reading, thanks for listening to my rant.

Best regards,
Dennis

On May 18, 2008, at 1:55 PM, Fred Tonetti wrote:

PS …

… and you REALLY don’t want to try this in AFL …

AB = CreateObject("Broker.Application");
AA = AB.Analysis;
AA.Backtest(0);
AA.Export("Trades.CSV");

… The reason of course is that whether you try and run a backtest with this AFL or just click the check mark to verify the AFL you’ll cause AB / AA to get into an infinitely running backtest i.e. a backtest which runs a backtest which runs a backtest ad infinitum …

However, if instead you create a .VBS file with this in it …

Set oAB = CreateObject("Broker.Application")
Set oAA = oAB.Analysis
oAA.Backtest(0)
oAA.Export("Trades.CSV")

Then the resulting Trades.CSV file will have what is expected in it.

I guess I have for a fair amount of time assumed that if what you need is directly available in AB, like ~~~Equity, then you can manipulate it directly in AFL … If what you need to get at is not directly accessible data, like a Trade List … Then you need a script to do this and by definition that script needs to be external to AB.

While I suppose it is conceivable to have an AFL that runs an EXTERNAL script which in turn runs a backtest and makes the results of what it does available to the calling AFL not only is that at least at first glance overkill and overly complicated it probably also fails for one reason or another …


From: amibroker@xxxxxxxxxps.com [mailto:amibroker@xxxxxxxxxps.com] On Behalf Of Fred Tonetti
Sent: Sunday, May 18, 2008 1:34 PM
To: amibroker@xxxxxxxxxps.com
Subject: RE: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?]

As a followup …

If you for example look at the AFL below, what it should produce and what it actually does produce you’ll see that:

-          The equity curve that results from running a backtest gets exported as one would think in should.
-          However, the output Trade List which is also a byproduct of running a backtest only exports the column headings when one attempts to do it via what would be OLE/Automation commands that more appropriately belong in something external to AB.

This is not a complaint per se, it’s simply what is …

Len1 = Optimize("Len1", 13, 1, 100, 1);
Len2 = Optimize("Len2", 23, 1, 100, 1);

EMA1 = EMA(C, Len1);
EMA2 = EMA(C, Len2);

Buy = Cross(EMA1, EMA2);
Sell = Cross(EMA2, EMA1);

OPEquity = Foreign("~~~Equity", "C");

Beg = LastValue(ValueWhen(Status("FirstBarInRange"), BarIndex()));
End = LastValue(ValueWhen(Status("LastBarInRange"), BarIndex()));

yyyy = Year();
mm   = Month();
dd   = Day();

fh   = fopen("C:\\Program Files\\AmiBroker\\Equity.CSV", "w");

fputs("Date,Equity,\n", fh);

for ( i = Beg; i <= End; i++ )

{
    Line = "";
    Line = Line + StrFormat("%02.0f", mm[i]) + "/" + StrFormat("%02.0f", dd[i]) + "/" + StrFormat("%4.0f", yyyy[i]) + ",";
    Line = Line + StrFormat("%.5f", OPEquity[i]) + ",";
    Line = Line + "\n";
    fputs(Line, fh);
}

fclose(fh);

AB = CreateObject("Broker.Application");
AA = AB.Analysis;
AA.Export("Trades.CSV");

From: amibroker@xxxxxxxxxps.com [mailto:amibroker@xxxxxxxxxps.com] On Behalf Of Fred Tonetti
Sent: Sunday, May 18, 2008 12:23 PM
To: amibroker@xxxxxxxxxps.com
Subject: RE: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?]

I see both sides of the discussion so maybe I can add something to bridge the gap …

While I agree with TJ that, as presented in the backtest report or AA in an optimization output line, system performance metrics are scalars i.e. single numbers as of the end of some backtest or some single combination of parameter values from an optimization, one could via the symbol ~~~Equity look at these as an array of values that can be evaluated bar by bar …

For example Net Profit has a value all along the array represented by the ~~~Equity symbol.

If one also needs to be able to have available Trade based performance metrics then one would also have to have available to them the list of Trades from a backtest so that as of some particular bar one could calculate whatever performance metric(s) one was interested in …

There are undoubtedly many uses for this sort of information one of which I touched on briefly the other day i.e. When during the OOS period should we programmatically decide that our system needs to be reoptimized and as result WF our IS to that point and do so …

While I have done these sorts of things that are described above in IO and other places, I would add that I have not for the most part done them directly in AFL because as TJ states you may very well not get what you are looking for.

If this is the kind of thing you are after it is not that difficult to do …

From an external script it is relatively simple to:

-          Run a backtest … This yields the Trade List which can be exported from AA to a CSV file.
-          It also yields the equity curve in the symbol ~~~Equity which can be easily exported via a “manufactured” AFL i.e. have the script create the AFL, load it and run it.
-          This then provides in two CSV files everything that one needs to calculate any and all performance metrics one is interested in bar by bar.
-          The results of these calculations can be used to drive indicators in AB or as all or part of a decision making process about what actions to perform in some other type of AFL in AB.

From: amibroker@xxxxxxxxxps.com [mailto:amibroker@xxxxxxxxxps.com] On Behalf Of Tomasz Janeczko
Sent: Sunday, May 18, 2008 10:58 AM
To: amibroker@xxxxxxxxxps.com
Subject: Re: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?]

You guys don't get it at all.

Most metrics are SCALARS (single number) !
Your plots will look like this:

Plot( 0.8, "Sharpe Ratio", colorRed );

These metrics DO NOT CHANGE over time because they are all calculated ONCE when backtest is complete
and result is single number (scalar), *not* array.

Best regards,
Tomasz Janeczko
amibroker.com
----- Original Message -----
From: Phsst
To: amibroker@xxxxxxxxxps.com
Sent: Sunday, May 18, 2008 3:36 PM
Subject: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?]

Herman,

Your case for System Performance Indicators is a good one.

A further idea... there should be no need for User code to accomplish this. The plots should be a native feature of AB (just like the Equity plot).

Instead of plotting "flat lines" for single numbers, gfx Horizontal plots of each performance metric could be presented against a "good / bad scale" defined in the form of a Param value for each performance metric that the User could alter from a base set of default values. Of course the gradient colors of the plots would go from red (bad) to green (good).

For example, the default Param for Sharpe Ratio could be 1.00, and some Users might want to use .80 as a Param value where the gradient color starts changing to green.

My two cents worth.

I am using the free version of SPAMfighter for private users.
It has removed 455 spam emails to date.
Paying users do not have this message in their emails.
Try SPAMfighter for free now!

I am using the free version of SPAMfighter for private users.
It has removed 455 spam emails to date.
Paying users do not have this message in their emails.
Try SPAMfighter for free now!

I am using the free version of SPAMfighter for private users.
It has removed 455 spam emails to date.
Paying users do not have this message in their emails.
Try SPAMfighter for free now!

I am using the free version of SPAMfighter for private users.
It has removed 455 spam emails to date.
Paying users do not have this message in their emails.
Try SPAMfighter for free now!

I am using the free version of SPAMfighter for private users.
It has removed 455 spam emails to date.
Paying users do not have this message in their emails.
Try SPAMfighter for free now!

I am using the free version of SPAMfighter for private users.
It has removed 455 spam emails to date.
Paying users do not have this message in their emails.
Try SPAMfighter for free now!

I am using the free version of SPAMfighter for private users.
It has removed 455 spam emails to date.
Paying users do not have this message in their emails.
Try SPAMfighter for free now!