Re: [amibroker] Re: statistivcs definition, AmiBroker Email List Archive

On Tue, Feb 9, 2010 at 9:00 PM, reefbreak_sd <reefbreak_sd@xxxxxxxxx> wrote:

I am no expert in backtesting, you need to address those questions to Howard Bandy on this forum or buy his EXCELLENT book "Quantitative Trading Systems" where he creates and backtests a variety of trading sysstems along with commentary. I understand Howard is coming out with an advanced version very soon. Also he has available a recording of a lecture he gave in Australia detailing the development and backtesting of a trading system. The cost is a trivial $1.95, but beware the lecture is "industrial strength".

You can limit the dollar size of each position and limit the number of positions you can hold, thereby removing the total dollars in the account as a determinant of profit.

Also note that there is several different levels of running the backtester. Allowing a very high level of control over what it does - This comes at a cost of complexity of programming.

You can add custom written calculations to the backtest report.

Reef

PS
I personally feel that backtesting is a "hall of mirrors" where you can easily fool yourself into thinking that you have a profitable system when you don't. Or conversely - needlessly discard a workable system because of poor backtest results. Be VERY skeptical of any results either good or bad.

--- In amibroker@xxxxxxxxxxxxxxx, Ted Byers <r.ted.byers@xxx> wrote:
>
> Thanks. That helped a lot.
>
> However, I am not especially happy with the following:
>
> System test report window

>
> correctly_annualized_perc_return = 100% * ( (final_value/initial_value) ^ (
> 365 / days_in_test ) - 1 )
>
> where x^y means rising x to the power of y
>
>
> The problem I find lays in knowing precisely what initial_value is supposed
> to be. Yes, the formula is correct, and makes sense, if the initial value
> is fully utilized in the investment (such as the initial and final value of
> a house one has purchased). However, in the system I am working on, at no
> point does the total value of all open positions exceed $100,000, and yet I
> have been instructed to set, as a default for backtesting purposes, initial
> cash at 1,000,000 (the trader we're working with doesn't want to deal with
> any limits posed by the available cash). But that means I can arbitrarily
> improve the rate of return by a factor of ten if I use an initial value of
> 100,000 or arbitrarily make it worse by using an initial value of
> 2,000,000. Obviously, I can't use an initial value less than 100,000 or the
> trades the system recommends would not be possible, but apart from that
> constraint, that I can arbitrarily alter initial value without affecting
> total return seems to make this figure just so much meaningless BS. At no
> time, when using this code, is position size computed from the available
> cash.
>
> How, then, can I define things in such a way that ensures that the rate of
> return actually has a meaningful value? So far, it seems I can only compute
> a meaningful value for the rate of return for a specific trade (where the
> initial value is obviously the cost of buying the shares, for a long
> position, or the value received from the sale of shares, for a short
> position), but not for a collection of trades distributed over a period of
> several years. Is there a standard way for handling this? One that is
> demonstrably valid?
>
>
> Thanks
>
>
> Ted
>
>
> On Mon, Feb 8, 2010 at 4:11 PM, reefbreak_sd <reefbreak_sd@xxx> wrote:
>
> >
> >
> > In AB, click on HELP -> SEARCH
> > type in "BackTester Report" in the search window
> > Under the "Select Topic" that comes back, click on "System Test Report
> > Window" and you will see a definition of each term. Many of these are
> > industry standard metrics of performance.
> >
> > You can go to www.investopedia.com and type in for example "risk adjusted
> > return" and get a more detailed explaination of most of the report items.
> >
> > Reef
> >
> >
> > --- In amibroker@xxxxxxxxxxxxxxx <amibroker%40yahoogroups.com>, Ted Byers

> > <r.ted.byers@> wrote:
> > >
> > > Hi All,
> > >
> > >
> > > I am digging into how AmiBroker computes the various statistics it
> > reports
> > > on a given back test.
> > >
> > > For example, I see the following reported:
> > >
> > > Exposure %
> > > Net Risk Adjusted Return %
> > > Annual Return %
> > > Risk Adjusted Return %
> > >
> > > Where will I find the details of exactly how AmiBroker computes these
> > from a
> > > given suite of trades? Now, I know what a risk adjusted return is, but in
> > > my previous work, it is based on 100% of the cash being invested, and it
> > > carries a specific definition of "risk". In the system I am working with
> > > now, it is rare for 100% of the available cash to be invested, and a
> > really
> > > really long lived trade lasts only a couple weeks (though 9 out of 10
> > trades
> > > were profitable in my last back test using my own C++ code - which in my
> > > view was quite bad - average position size of about 50,000 and total
> > profit
> > > over 2 years being about 10,000). It seems rather meaningless to report
> > an
> > > annual rate of return if one is in the market for less than a month.
> > >
> > > In this "system", the position size is not a function of the total amount
> > of
> > > cash available; so if these rates are defined relative to the total
> > amount
> > > of cash available, I can arbitrarily change these rates by increasing or
> > > decreasing the amount of cash available, within limits. I am afraid my
> > > understanding of these ideas is derived from the perspective of an
> > investor
> > > (where one buys a stock, and looks at returns based on the combination of
> > > dividends and long term change in share value, adjusted for splits, &c.).
> > > If I can see how AmiBroker computes these stats for a given backtest, I
> > hope
> > > to understand these from the perspective of a trader.
> > >
> > > So where will I find the documentation of how AmiBroker computes these
> > back
> > > test statistics?
> > >
> > > Thanks
> > >
> > > Ted
> > >
> >
> >
> >
>
>
>
> --
> R.E.(Ted) Byers, Ph.D.,Ed.D.
> TED@xxx

> CTO
> Merchant Services Corp.
> 350 Harry Walker Parkway North, Suite 8
> Newmarket, Ontario
> L3Y 8L3
>