[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [Metastockusers] System Tests alternatives



PureBytes Links

Trading Reference Links

Title: Message
Thanks for the detailed run down of how you go about things Super.  It was interesting to read. I don't actually use MS, but do my testing in TS and WL.  Unlike you I try to make my systems as realistic as possible so that when the real results come thorough I can compare it to the simulation.  If you exit based on the condition of the market then I will simply make that part of the system. Why test something that your not going to trade apart from initially trying to get a generally feel as to whether the idea has merit.  Once you prove it has, then you refine it to the point where the simulation IS the way you will trade it...assuming you want to trade totally mechanically. Perhaps this is easier to do for me as I have had many years of trading experience and know what assumptions are realistic and what are not. 
 
Adrian
-----Original Message-----
From: Metastockusers@xxxxxxxxxxxxxxx [mailto:Metastockusers@xxxxxxxxxxxxxxx] On Behalf Of superfragalist
Sent: Saturday, 16 April 2005 2:00 AM
To: Metastockusers@xxxxxxxxxxxxxxx
Subject: [Metastockusers] System Tests alternatives


Adrian,

I use the systems tester to make gross approximations. All systems
tests are only good on a relative basis. You can compare one MS
systems test to another MS systems test and you can rank them. You
shouldn't compare an MS systems test to trade station systems test.

Say you test five different systems and then you rank them in the
order of their performance.

That ranking is based on what you see on the statistical side and not
whether a system is any good in live trading. It's hard to determine
how close someone will get to the systems tests statistics until they
analyze data from live trading.

To avoid surprises, I run systems through the systems tester, and if
they look like something that fits within the parameters of how I
trade, I write an exploration from the same equations.

I have predefined historical time periods where I know what kind of
market behavior was occurring. I do several (10 to 100 bar by bar)
expplorations during a particular market condition. I open the charts,
apply my trading rules to the charts and create a list of the trades I
would have taken. I then look at the chart to see how the trades I
would have taken performed.

This way I can see how the system works in up trends, down trends,
sideways markets, etc.

If there is something that I don't like that I'm seeing, like too many
charts passing the exploration, too few, or a variety of other things
I've learned to avoid, I check the equations to see if I can fix the
problem with filters or with other adjustments to it.

Once I'm satisfied that a system has a good chance of working for me
in live trading, I paper trade it for a couple of months sometimes
longer.

I have a program that tracks those results.

The actual results are never the same, or really even close to the
same, as those in the tester. However, if I have ranked 5 systems with
the tester, the rankings after these explorer/trading tests have
tended to be similar, even though the numerical values are not.

There are many many variables that can not be put into a systems
tester. I want the chart to show certain characteristics before I take
a trade, but my chart "look" is not precise and is difficult to
duplicate with equations.

Another issue that impacts systems tests is the choice of exits and
stops. The system tester is consistent in applying those. I'm not. I
change my exits and stops according to market conditions. This is
another reason the systems tester is only an approximation and a crude
one at that.

Over the years, I've run many thousands of systems tests and put
probably 100 different systems through the explorer side of the tests.
Out of that I have found a few systems that fit my trading profile.
Those systems are the ones I rely on.

In response to Andrew's comments about keeping track of slippage.
That's the only way to do it. I don't bother putting slippage into my
systems tests or commissions. My actual slippage depends on many
factors. If a symbol is a high volume stock like C, then slippage is
not a factor. If it's a master limited partnership, or REIT, slippage
is larger, but the holding period tends to be longer.

My trade clearing is now so much faster than it used to be, I don't
consider slippage to be a major pain like it once was. The commissons
have dropped down to the decimal places.











--- In Metastockusers@xxxxxxxxxxxxxxx, "Adrian Pitt" <apitt@xxxx> wrote:
> If you believe in a few core trading methods, presumably from
> simulations you have done applying them to the type of market conditions
> they are suited to, but at the same time you say you don't rely on
> system tests....so I'm fascinated...what do you rely on if you don't
> have extensive real time results?  How do the above two views possibly
> sit side by side?

> Regards,
> Adrian
>
> -----Original Message-----
> From: Metastockusers@xxxxxxxxxxxxxxx
> [mailto:Metastockusers@xxxxxxxxxxxxxxx] On Behalf Of superfragalist
> Sent: Friday, 15 April 2005 12:49 PM
> To: Metastockusers@xxxxxxxxxxxxxxx
> Subject: [Metastockusers] Re: System Tester Results (Redu) Idea for
> Scott
>
>
>
> Slippage is often a function of the vendor you are using.
>
> I use Fidelity. I'm a big block trader and I have very little slippage
> and very fast execution. It wasn't that way when I first started with
> them. I had several conversations with their order desk, and over time
> they've made huge improvements. There trading software is now also
> excellent. At first it was pretty but not real good. Now it is as good
> as any trade execution software I've used, and I've tried a lot of them.
>
>
> Your idea about narrowing your list of stocks is a good idea but when
> I talk about narrowing the list, I'm talking about a universe of maybe
> 100 to 200 stocks. A lot of my trading buddies keep their lists to
> less than 20.
>
> I don't rely much on systems tests results because I've never seen a
> system that produced nearly the same stats in live trading. Perhaps it
> depends on whose is driving the boat.
>
> In addition, there are a lot of settings I can make with the systems
> tester that gives me results that can't be duplicated in living trading.
>
>
> Anyway, have fun!
>
> Don't forget us little people when you get wealthy. 
>
>
>
>
>
> --- In Metastockusers@xxxxxxxxxxxxxxx, "Scooter" <mariani@xxxx> wrote:
> > Super,
> > I am only cautiously optimistic I can make any money with this system.
> I
> > have been paper trading it this month with decent results but I
> don't think
> > I am capturing slippage properly. I need some education on placing
> large
> > (for me) orders.
> >
> > I will go back and search the forum for documented differences
> between 8.0
> > and 8.01.
> >
> > Thanks again for taking the time to try and help me and others, I
> know how
> > valuable time is.
> > Scott
> >   -----Original Message-----
> >   From: Metastockusers@xxxxxxxxxxxxxxx
> > [mailto:Metastockusers@xxxxxxxxxxxxxxx]On Behalf Of superfragalist
> >   Sent: Thursday, April 14, 2005 9:21 PM
> >   To: Metastockusers@xxxxxxxxxxxxxxx
> >   Subject: [Metastockusers] Re: System Tester Results (Redu) Idea
> for Scott
> >
> >
> >
> >   Okay, good luck with the system. In three years, you can tell us how
> >   much you've made. Get the results audited, and you can sell the
> system
> >   to the less fortunate.
> >
> >   Next time you run tests year by year use the calendar year. I've run
> >   into some problems with test results that used starting dates in the
> >   middle of the year. Intuitively, it doesn't seem like it would
> matter,
> >   but I've seen a few systems tests where it did make a difference.
> >
> >   --- In Metastockusers@xxxxxxxxxxxxxxx, "Scooter" <mariani@xxxx>
> wrote:
> >   > Roy,
> >   > I was afraid of that.
> >   >
> >   > Super,
> >   > Thanks for the detailed explanation. Guess I am SOL.
> >   >
> >   > Here are the overall portfolio profits each year for 10 years.
> Based on
> >   > $20,000 initial with 100% available equity invested.
> >   >
> >   >       Test # Description OPT1=2 OPT1=3
> >   >       Simulation 26
> >   >       Simulation 33 Tickers from 4/13/95 to 4/13/96 $56,191.43
> >   $50,938.91
> >   >       Simulation 34 Tickers from 4/13/96 to 4/13/97 $57,085.49
> >   $16,903.62
> >   >       Simulation 35 Tickers from 4/13/97 to 4/13/98 $77,511.63
> >   $114,957.57
> >   >       Simulation 36 Tickers from 4/13/98 to 4/13/99 $71,420.85
> >   $98,303.31
> >   >       Simulation 37 Tickers from 4/13/99 to 4/13/00 $209,405.87
> >   $290,114.33
> >   >       Simulation 38 Tickers from 4/13/00 to 4/13/01 $1,615.60
> >   ($29,693.35)
> >   >       Simulation 39 Tickers from 4/13/01 to 4/13/02 $152,639.93
> >   $158,116.91
> >   >       Simulation 40 Tickers from 4/13/02 to 4/13/03 ($11,963.12)
> >   $106,714.19
> >   >       Simulation 41 Tickers from 4/13/03 to 4/13/04 $160,324.40
> >   $234,187.23
> >   >       Simulation 42 Tickers from 4/13/04 to 4/13/05 $85,159.69
> >   $139,048.32
> >   >       $859,391.77  $1,179,591.04
> >   >
> >   >
> >   > I don't have the time to manually move the individual data to
> excel
> >   at this
> >   > time.
> >   >
> >   > Thanks again, Scott
> >   >   -----Original Message-----
> >   >   From: Metastockusers@xxxxxxxxxxxxxxx
> >   > [mailto:Metastockusers@xxxxxxxxxxxxxxx]On Behalf Of Roy Larsen
> >   >   Sent: Thursday, April 14, 2005 8:50 PM
> >   >   To: Metastockusers@xxxxxxxxxxxxxxx
> >   >   Subject: Re: [Metastockusers] Re: System Tester Results (Redu)
> >   Idea for
> >   > Scott
> >   >
> >   >
> >   >   Scott
> >   >
> >   >
> >   >   Somewhere I got the impression you were using MS 8.0. I may be
> >   wrong but I
> >   > don't think it's possible
> >   >   to copy System Tester results from 8.0. I think that was one of
> >   the bugs
> >   > fixed with 8.01. The "copy
> >   >   to clipboard" option certainly works for 9.0, though Ctrl+C
> >   doesn't seem
> >   > to work. Once pasted to the
> >   >   clipboard Crtl+V works OK for pasting into Excel.
> >   >
> >   >
> >   >   Kind regards
> >   >
> >   >   Roy Larsen
> >   >   www.metastocktips.co.nz
> >   >   Free formulas and MS links
> >   >
> >   >
> >   >
> >   >
> >   >
> >   >   ----- Original Message -----
> >   >   From: "Scooter" <mariani@xxxx>
> >   >   To: <Metastockusers@xxxxxxxxxxxxxxx>
> >   >   Sent: Friday, April 15, 2005 11:15 AM
> >   >   Subject: RE: [Metastockusers] Re: System Tester Results (Redu)
> >   Idea for
> >   > Scott
> >   >
> >   >
> >   >   > If only it was that easy. I can not get the data out of MS,
> short of
> >   >   > manually transferring every symbol. I select the results but
> the
> >   only
> >   > right
> >   >   > mouse click option is "discard". I have tried all the Windows
> >   tricks I
> >   > know,
> >   >   > no luck.
> >   >   > Scott
> >   >   >  -----Original Message-----
> >   >   >  From: Metastockusers@xxxxxxxxxxxxxxx
> >   >   > [mailto:Metastockusers@xxxxxxxxxxxxxxx]On Behalf Of John
> >   >   >  Sent: Thursday, April 14, 2005 6:19 PM
> >   >   >  To: Metastockusers@xxxxxxxxxxxxxxx
> >   >   >  Subject: [Metastockusers] Re: System Tester Results (Redu)
> Idea for
> >   > Scott
> >   >   >
> >   >   >
> >   >   >
> >   >   >  I'm sorry Scott, I meant to say copy results from the system
> >   tester,
> >   >   >  not the explorer. You open your test results page and follow
> >   the steps
> >   >   >  I've explained. Highlight the results you want to transfer to
> >   excel,
> >   >   >  copy to the clipboard, etc.
> >   >   >
> >   >   >
> >   >   >
> >   >   >
> >   >   >
> >   >   >  --- In Metastockusers@xxxxxxxxxxxxxxx, "Scooter"
> <mariani@xxxx>
> >   wrote:
> >   >   >  > Super,
> >   >   >  > Thanks for taking some time to help me!
> >   >   >  >
> >   >   >  > I know how to get the results from the Explorer, I do not
> >   know how
> >   >   >  to get
> >   >   >  > the results from the System Tester.
> >   >   >  > I am excited to try what you suggest but need a little more
> >   help to
> >   >   >  start.
> >   >   >  >
> >   >   >  > Scott
> >   >   >  >   -----Original Message-----
> >   >   >  >   From: superfragalist [mailto:jackolso@xxxx]
> >   >   >  >   Sent: Thursday, April 14, 2005 11:05 AM
> >   >   >  >   To: Metastockusers@xxxxxxxxxxxxxxx
> >   >   >  >   Subject: [Metastockusers] System Tester Results (Redu)
> Idea for
> >   > Scott
> >   >   >  >
> >   >   >  >
> >   >   >  >
> >   >   >  >   Scott,
> >   >   >  >
> >   >   >  >   Here's an idea that you can use to test your theory. I
> >   explained
> >   > how
> >   >   >  >   to copy results from an exploration into excel and then
> >   from excel
> >   >   >  >   back into various MS lists.
> >   >   >  >
> >   >   >  >   Run your systems test year by year on your expert.
> >   >   >  >
> >   >   >  >   Sort the test alphabetically, and then copy the entire
> test for
> >   > each
> >   >   >  >   year into a new excel worksheet for that year.
> >   >   >  >
> >   >   >  >   From each worksheet copy the symbol and it's annual
> percentage
> >   > return
> >   >   >  >   into a new worksheet. Once you have delete symbols that
> >   don't match
> >   >   >  >   across all ten columns, you can delete all of the symbols
> >   except
> >   > the
> >   >   >  >   first year's. You'll be left with a year by year
> comparison
> >   of the
> >   >   >  >   return on each symbol with your expert.
> >   >   >  >
> >   >   >  >   You can then see which symbols were consistently
> profitable and
> >   > which
> >   >   >  >   one weren't. This will answer your question.
> >   >   >  >
> >   >   >  >
> >   >   >  >
> >   >   >  >
> >   >   >  >
> >   >   >  >   --- In Metastockusers@xxxxxxxxxxxxxxx, "Scooter"
> <mariani@xxxx>
> >   > wrote:
> >   >   >  >   > Super,
> >   >   >  >   > I did read Mr. Wright's series of articles as you
> suggested
> >   >   >  >   previously. I
> >   >   >  >   > found it very informative. I found that I was already
> >   doing many
> >   >   >  of the
> >   >   >  >   > things he suggested in developing a system. I have
> >   applied many
> >   > of
> >   >   >  >   the ideas
> >   >   >  >   > he talked about since reading the articles, more like
> a book!
> >   >   >  >   >
> >   >   >  >   > He does not address the validity of throwing out
> losers,
> >   based on
> >   > a
> >   >   >  >   system
> >   >   >  >   > test, from an available list of potential stocks to
> trade.
> >   >   >  >   >
> >   >   >  >   > I know there are a cagillion system developing books to
> >   read. I
> >   > will
> >   >   >  >   pick up
> >   >   >  >   > a few to read when I have the time.
> >   >   >  >   >
> >   >   >  >   > Can you C&P system test results from MS8.0?
> >   >   >  >   >
> >   >   >  >   > Scott
> >   >   >  >   >
> >   >   >  >   >
> >   >   >  >   >  -----Original Message-----
> >   >   >  >   > From: superfragalist [mailto:jackolso@xxxx]
> >   >   >  >   > Sent: Wednesday, April 13, 2005 8:34 PM
> >   >   >  >   > To: Metastockusers@xxxxxxxxxxxxxxx
> >   >   >  >   > Subject: [Metastockusers] Re: System Tester Results
> (Redu)
> >   >   >  >   >
> >   >   >  >   >
> >   >   >  >   >
> >   >   >  >   >   Looks like you're using the systems tester without
> much
> >   >   >  knowledge of
> >   >   >  >   >   systems development.
> >   >   >  >   >
> >   >   >  >   >   Read a few books on system design and then you'll see
> >   you're
> >   >   >  wasting
> >   >   >  >   >   your time.
> >   >   >  >   >
> >   >   >  >   >   There's a very good article on this coming up in the
> >   May issue
> >   > of
> >   >   >  >   >   Roy's newsletter.
> >   >   >  >   >
> >   >   >  >   >   Don't ask for book recommendations because one book
> >   won't do
> >   > it.
> >   >   >  >   >   You'll need to read around a dozen or so systems
> >   development
> >   >   >  books to
> >   >   >  >   >   get a really good take on how things work. Once
> you've done
> >   > that,
> >   >   >  >   >   you'll soon figure out that using TA by itself is not
> >   the best
> >   >   >  method
> >   >   >  >   >   for picking stocks to trade, the performance of any
> system
> >   >   >  based on TA
> >   >   >  >   >   is going to dramatically change each year with a lot
> of
> >   losing
> >   >   >  years,
> >   >   >  >   >   there are no indicators that work even reasonably
> well
> >   across a
> >   >   >  >   >   variety of stocks or in changing market conditions,
> and
> >   > searching
> >   >   >  >   >   thousands of stocks every night for something to
> trade
> >   is in
> >   >   >  the long
> >   >   >  >   >   run a losing idea.
> >   >   >  >   >
> >   >   >  >   >   Does that mean you can't make money using TA. You
> can,
> >   but you
> >   >   >  have to
> >   >   >  >   >   have a well thought out strategy and apply the
> strategy
> >   >   >  appropriately
> >   >   >  >   >   for the market conditions.
> >   >   >  >   >
> >   >   >  >   >   I'm amazed at how many people that refuse to spend a
> >   dime on
> >   >   >  >   >   education, information, good data or anything else
> that
> >   might
> >   > help
> >   >   >  >   >   them, but they'll pay a guru thousands for useless
> trading
> >   >   >  methods or
> >   >   >  >   >   lose thousands more making bad trades. It's
> baffaling.
> >   >   >  >   >
> >   >   >  >   >   You can read Charlie Wright's series of articles
> called
> >   >   >  Trading as a
> >   >   >  >   >   Business. That will give you some basics about how to
> >   approach
> >   >   >  this.
> >   >   >  >   >   Read all the articles in the series. They all have
> >   something
> >   >   >  important
> >   >   >  >   >   to say.
> >   >   >  >   >
> >   >   >  >   >   http://www.elitetrader.com/tr/index.cfm?s=17
> >   >   >  >   >
> >   >   >  >   >   Give Roy's newsletter a try. It's all about MS with
> the
> >   code
> >   > for
> >   >   >  >   >   everything discussed in it. You'll learn more from
> that
> >   in a
> >   >   >  few hours
> >   >   >  >   >   of reading the back issues than you will from playing
> >   with the
> >   >   >  systems
> >   >   >  >   >   tester for next few years.
> >   >   >  >   >
> >   >   >  >   >   www.metastocktips.co.nz
> >   >   >  >   >
> >   >   >  >   >
> >   >   >  >   >
> >   >   >  >   >
> >   >   >  >   >
> >   >   >  >   >
> >   >   >  >   >
> >   >   >  >   >   --- In Metastockusers@xxxxxxxxxxxxxxx, "Scooter"
> >   >   >  <mariani@xxxx> wrote:
> >   >   >  >   >   > O.k.,
> >   >   >  >   >   > I have finally cooked the indicators enough
> (nothing
> >   new) to
> >   >   >  come up
> >   >   >  >   >   with a
> >   >   >  >   >   > system that is profitable over several different
> >   period back
> >   >   >  tests,
> >   >   >  >   >   at least
> >   >   >  >   >   > in MS. My question is, do I remove the loosing or
> >   zero trade
> >   >   >  tickers
> >   >   >  >   >   from my
> >   >   >  >   >   > potential list of tradable tickers I will run my
> >   Exploration
> >   >   >  on when
> >   >   >  >   >   looking
> >   >   >  >   >   > for a new trading opportunity? MS seems to be keen
> on
> >   > "cleaning
> >   >   >  >   up" by
> >   >   >  >   >   > removing the losers. Doesn't make sense to me to
> say
> >   because
> >   > the
> >   >   >  >   >   tickers did
> >   >   >  >   >   > not hit or make a profit over x amount of periods
> >   they will
> >   > not
> >   >   >  >   make a
> >   >   >  >   >   > profit in the future. Or does it increase my
> potential
> >   >   >  >   profitability by
> >   >   >  >   >   > removing them? Can I somehow export the remaining
> ticker
> >   >   >  symbols to
> >   >   >  >   >   excel or
> >   >   >  >   >   > csv so I can adjust my HSQuote list accordingly?
> >   >   >  >   >   > Thanks, Scott
> >   >   >  >   >
> >   >   >  >   >
> >   >   >  >   >
> >   >   >  >   >
> >   >   >  >   >
> >   >   >  >   >
> >   >   >  >
> >   >
> >   > >
> > 
> ------------------------------------------------------------------------
> -
> >   > -
> >   >   >  > --
> >   >   >  >   > --
> >   >   >  >   >   Yahoo! Groups Links
> >   >   >  >   >
> >   >   >  >   >     a.. To visit your group on the web, go to:
> >   >   >  >   >     http://groups.yahoo.com/group/Metastockusers/
> >   >   >  >   >
> >   >   >  >   >     b.. To unsubscribe from this group, send an email
> to:
> >   >   >  >   >     Metastockusers-unsubscribe@xxxxxxxxxxxxxxx
> >   >   >  >   >
> >   >   >  >   >     c.. Your use of Yahoo! Groups is subject to the
> >   Yahoo! Terms
> >   > of
> >   >   >  >   Service.
> >   >   >  >
> >   >   >  >
> >   >   >  >
> >   >   >  >
> >   >   >  >
> >   >   >  >
> >   >
> >   > >
> > 
> ------------------------------------------------------------------------
> -
> >   > -
> >   >   > --
> >   >   >  > --
> >   >   >  >   Yahoo! Groups Links
> >   >   >  >
> >   >   >  >     a.. To visit your group on the web, go to:
> >   >   >  >     http://groups.yahoo.com/group/Metastockusers/
> >   >   >  >
> >   >   >  >     b.. To unsubscribe from this group, send an email to:
> >   >   >  >     Metastockusers-unsubscribe@xxxxxxxxxxxxxxx
> >   >   >  >
> >   >   >  >     c.. Your use of Yahoo! Groups is subject to the Yahoo!
> >   Terms of
> >   >   >  Service.
> >   >   >
> >   >   >
> >   >   >
> >   >   >
> >   >   >
> >   >
> >   >
> >   > >
> > 
> ------------------------------------------------------------------------
> --
> >   > --
> >   >   > --
> >   >   >  Yahoo! Groups Links
> >   >   >
> >   >   >    a.. To visit your group on the web, go to:
> >   >   >    http://groups.yahoo.com/group/Metastockusers/
> >   >   >
> >   >   >    b.. To unsubscribe from this group, send an email to:
> >   >   >    Metastockusers-unsubscribe@xxxxxxxxxxxxxxx
> >   >   >
> >   >   >    c.. Your use of Yahoo! Groups is subject to the Yahoo!
> Terms of
> >   > Service.
> >   >   >
> >   >   >
> >   >
> >   >
> >   >
> >   >
> >   >
> > 
> ------------------------------------------------------------------------
> --
> > --
> >   > --
> >   >   Yahoo! Groups Links
> >   >
> >   >     a.. To visit your group on the web, go to:
> >   >     http://groups.yahoo.com/group/Metastockusers/
> >   >
> >   >     b.. To unsubscribe from this group, send an email to:
> >   >     Metastockusers-unsubscribe@xxxxxxxxxxxxxxx
> >   >
> >   >     c.. Your use of Yahoo! Groups is subject to the Yahoo! Terms
> of
> >   Service.
> >
> >
> >
> >
> >
> >
> ------------------------------------------------------------------------
> ----
> > --
> >   Yahoo! Groups Links
> >
> >     a.. To visit your group on the web, go to:
> >     http://groups.yahoo.com/group/Metastockusers/
> >
> >     b.. To unsubscribe from this group, send an email to:
> >     Metastockusers-unsubscribe@xxxxxxxxxxxxxxx
> >
> >     c.. Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> Service.
>
>
>
>
>
>   _____ 
>
> Yahoo! Groups Links
>
>
> *      To visit your group on the web, go to:
> http://groups.yahoo.com/group/Metastockusers/
>  
>
> *      To unsubscribe from this group, send an email to:
> Metastockusers-unsubscribe@xxxxxxxxxxxxxxx
> <mailto:Metastockusers-unsubscribe@xxxxxxxxxxxxxxx?subject=Unsubscribe>
>  
>
> *      Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> Service <http://docs.yahoo.com/info/terms/> .





Yahoo! Groups Links