[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[EquisMetaStock Group] Re: Systems Testing Report metrics - your opinion please



PureBytes Links

Trading Reference Links


Hi Andrew,

> So I confess to being curious. What kind of trend filters are you
guys using
> that are backtestable, verifiable and timely, and enable you to
profitably
> switch between models? Heck, if I knew when the market was starting and
> ending a trend, I wouldn't need a model, I'd just go 100% leveraged
in the
> trend direction. 

Depends on the market.....  As mentioned, an individual model is
picked for each market.  The trend filter is not used to switch, it is
a model that is used on its own, along with the rest.  See below on
how to switch.

> 2. "If you know the market is going into a trending phase, switch
over to
> your trending system, if you know volatility is picking up, stop
using your
> trending system e.g." we've been hearing a lot of this recently. Use
your
> trending model when the market is trending, use your sideways model
when it
> isn't, etc. Which begs the question, how do you decide what kind of
market
> you are in in a timely and consistent fashion? Even the 100% trend
followers
> reckon they only catch around 1/3 of the trend, and that's with
models that

Note that we use, for one, volatility to decide.  Also, it is not the
on/off switch that you refer to somewhere else, but a weight that is
increased or decreased.  Thus the model is still used, just more or
less of it, rather than switching it on or off and switching to a
different model.  If you jump from one model to the other you're
asking for trouble.  Techniques to do this (adjust weights) include
the Kalman filter and neural networks, as mentioned before and
discussed in previous posts.

One of the best ways to know if you are in a trend or not, is to use
your trend model!  Very simply, if it starts to fail, then you know
the market is not trending anymore.  Store and evaluate model errors
on a continous basis.  So even a bad model (at the time) gives good
information in that it is not working.  This may not be as timely as
the turning point, but, if you pay close attention, will be timely enough.

Regards
MG Ferreira
TsaTsa EOD Programmer and trading model builder
http://www.ferra4models.com
http://fun.ferra4models.com 





--- In equismetastock@xxxxxxxxxxxxxxx, "Andrew Tomlinson"
<andrew_tomlinson@xxxx> wrote:
> 1. There has been occasional discussion on performance metrics on
this board
> - search and you will find. If you want to get serious about metrics you
> have to export results out of MetaStock (or better, TradeSim) and
run your
> own statistics in Excel. 
> 
> 2. "If you know the market is going into a trending phase, switch
over to
> your trending system, if you know volatility is picking up, stop
using your
> trending system e.g." we've been hearing a lot of this recently. Use
your
> trending model when the market is trending, use your sideways model
when it
> isn't, etc. Which begs the question, how do you decide what kind of
market
> you are in in a timely and consistent fashion? Even the 100% trend
followers
> reckon they only catch around 1/3 of the trend, and that's with
models that
> are only trying to play trends  i.e. that catch every one. If you go
back
> and just test your models against periods of time when you know in
> retrospect that there was a trend, of course they're going to look
good. But
> it's hindsight. And its unverifiable. 
> 
> The latest TASC has an article on trading pullbacks that does the same
> thing. Writes a trend-following system and tests it against past periods
> where trends were identifiable by trendline breaks. Great in
retrospect, but
> unverifiable (unless you can exactly describe your trendline
criteria in a
> way that MS can understand). It may be a great system, but its not back
> testable as described.
> 
> I can understand running several models at the same time so that
something
> is always working, which should give you low-risk results. But I
understand
> we're being told we should be able to identify trading environments
early
> enough to switch models entirely.
> 
> So I confess to being curious. What kind of trend filters are you
guys using
> that are backtestable, verifiable and timely, and enable you to
profitably
> switch between models? Heck, if I knew when the market was starting and
> ending a trend, I wouldn't need a model, I'd just go 100% leveraged
in the
> trend direction. 
> 
> OK, I'm salivating now. Someone put me out of my misery. (And just
to do a
> little filtering of responses, I've done quite a bit of backtesting
on very
> large stock portfolios in TradeSim on some of the traditional trend
filters,
> ADX, MA combos, Aroon, etc. over long time periods and didn't identify
> results much different from random - so I mostly focus on exit
technologies,
> as they seem to have a little more impact)
> 
> Andrew
> 
> 
> 
> -----Original Message-----
> From: MG Ferreira [mailto:quant@x...] 
> Sent: Monday, April 11, 2005 2:19 AM
> To: equismetastock@xxxxxxxxxxxxxxx
> Subject: [EquisMetaStock Group] Re: Systems Testing Report metrics -
your
> opinion please
> 
> 
> 
> 
> Hi there,
> 
> We optimise and test many, many systems on a variety of markets. 
> Currencies, commodities, stocks - you name it, we test it.  What we have
> found to work in practise contains some elements of the following:
> 
> - There is no universal system working in all markets.  A good
system will
> have some 'universal' attributes and there is a lot of 'spill over'
from one
> market into another, but a mechanical system must be built for a given
> market and kept up to date vigorously for this market.
> 
> - Even for a single market, try and use a lot of systems.  You have
to make
> the final choise but if you have a large playing field it helps.  If you
> know the market is going into a trending phase, switch over to your
trending
> system, if you know volatility is picking up, stop using your trending
> system e.g.
> 
> - Even though there is emotion in it, try to add your own 'gutt
feel' to the
> decision making process.  We often use discression to give more or less
> weight to a given system based on our own, subjective reading of the
> fundamentals.
> 
> - If you require a completely automated system, things will get
complex and
> you have to prepare for this.  We have built such systems, but to take a
> human out of it completely adds many things that otherwise are taken
care of
> easily.  Scaling in and out, sometimes ignoring the model until an event
> works through the system, roll-over and expiry of contracts - humans
can do
> these things much easier than computers.
> 
> In terms of your question specifically.  I haven't read any of those
books,
> but we use a variety of metrics.  We look at overall profit, we look at
> profit in a hold-out period (you should use hold-out periods to properly
> test a system), we adjust profits in all sorts of ways and finally,
pretty
> much in line with the above, we make up a metric by weighing a number of
> these metrics.  I think we have eight metrics, which we combine in a
single,
> ninth metric and this is what we then use as our metric to pick systems.
> But sometimes, for a specific task (such as least risky or most
active) we
> may switch to one of the individual metrics.
> 
> We also don't use MS anymore to test systems.  MS has many
limitations and
> we developed our own system tester a long time ago to overcome these and
> have been much happier ever since.  MS is good for exploration,
testing of
> concepts, prototyping and so on, but it has its limits as well.
> 
> Regards
> MG Ferreira
> TsaTsa EOD Programmer and trading model builder
http://www.ferra4models.com
> http://fun.ferra4models.com 
> 
> 
> 
> 
> 
> 
> --- In equismetastock@xxxxxxxxxxxxxxx, superfragalist <no_reply@xxxx>
> wrote:
> > 
> > I've read the Stridsman's and Conway's books and basically there
isn't 
> > any system in them that an individual can apply and make much from. I 
> > liked the books. Stridsman tells you a lot about what won't work. And 
> > my own test results confirm his.
> > 
> > Stridsman's systems are very hard to program in MS. However, I've 
> > tested a few to them and frankly there are easier and higher 
> > probability ways to trade. Stridsman methods do find some good
trades, 
> > but not too many and not too often.
> > 
> > Roy has a very good discussion of mechanical systems and systems
where 
> > you actually have to make a decision.
> > 
> > There a few simple methods of trading that will make most individuals 
> > far more money than a mechanical system.
> > 
> > However, if you have time, money and programming ability you might be 
> > able to come up with a half decent mechanical system. I've tested 
> > something over 500 of them and so far nothing comes close to the 
> > trading methods I use.
> > 
> > I've found that mechanical systems work better for hedge funds and 
> > funds where there is a large amount of money under management and
they 
> > have to trade multiple markets to deploy it. In those cases they have 
> > the money and time to handle the huge drawdowns of a mechancial 
> > system. They're executing based on many mechanical systems at the
same 
> > time, when the system starts producing poor results they can discard 
> > it and try something else, and their trades are carried out by
> computer. 
> > 
> > Active Trader Magazine tests hundreds of so called mechanical
systems. 
> > I've taken the best one's they've come up with and tested them on a 
> > larger market sample than they use. The S&P 500. All of their systems 
> > looked bad when tested on a larger market segment. Even their good 
> > systems had huge drawdowns.
> > 
> > If you don't have the emotional ability to look at a chart and buy
the 
> > stocks based on your judgement, you won't be able to stomach the 
> > drawdowns of most mechanical systems.
> > 
> > Future's Truth magazine has tested and tracked the performance of
many 
> > mechanical systems over the years. While they've found a few that
work 
> > reasonably well, most of the time, when an individual starts using 
> > them, they second guess the system and the actual live trading
results 
> > are far worse than the tests.
> > 
> > There are a lot of people on here that will disagree with me, and 
> > that's fine. If they want to use a mechanical system and they're
happy 
> > with the results, get them to give you their system, and you can both 
> > trade it. I've just never found one that I thought was worth much.
> > 
> > You have to remember if there was a mechanical system making any 
> > serious money a whole lot of people would be using it. Think about
all 
> > the smart guys with unlimited amounts of computer time and money who 
> > have been chasing just such a thing.
> > 
> > The simple trading methods outlined in Roy's newsletter work better 
> > for me than all of the systems I've tested to this point. However, I 
> > have an open mind and as soon as something comes along that beats
what 
> > I'm doing now, I'm on it.
> > 
> > www.metastocktips.co.nz
> > 
> > All the newbie's think they're going to discover some great new 
> > system, or copy some guru's crap and the money is going to roll in.
> > 
> > Let me know when that happens.
> > 
> > 
> > 
> > 
> > 
> > --- In equismetastock@xxxxxxxxxxxxxxx, "Todd" <todd@xxxx> wrote:
> > > 
> > > 
> > > Has any one read Conway's Professional Stock Trading or s 
> > > Stridsman's
> > > Trading Systems That Work?
> > >  
> > > I also have a question about the value of some of the metrics in the
> > > Systems Testing reports.  Which do you find most valuable?  I
believe 
> > > that some probably should not be used at all as they are 
> > > either "meaningless" or could be misleading.  Of course, I may not 
> > > understand them so that could be a problem, but after reading a few 
> > > papers on trading systems metrics, this is what I have concluded.
> > >  
> > > What metrics do you use in addition to those on the reports?
> > >  
> > > Thanks!
> 
> 
> 
> 
> 
> 
>  
> Yahoo! Groups Links





------------------------ Yahoo! Groups Sponsor --------------------~--> 
In low income neighborhoods, 84% do not own computers.
At Network for Good, help bridge the Digital Divide!
http://us.click.yahoo.com/EpW3eD/3MnJAA/cosFAA/BefplB/TM
--------------------------------------------------------------------~-> 

 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/equismetastock/

<*> To unsubscribe from this group, send an email to:
    equismetastock-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/