[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: System Testing Validity



PureBytes Links

Trading Reference Links

Steve & Yuki,

The two of you 'filled in the blanks' for me regarding trading a
subset of stocks.

It never occurred to me to take a mech stradegy that worked 'OK' on
the entire market, and then extract the best performing subset for
trading purposes.

I've been doing this (trading backtested results) for years
(successfully) and never considered what you guys are suggesting.

Thanks for teaching an 'old dog', 'new tricks'.

Phsst



 In amibroker@xxxxxxxxxxxxxxx, "CedarCreekTrading" <kernish@xxxx> wrote:
> Isn't there any worry that by restricting a mech system to
> a small basket of selected stocks that you might be doing some extreme
> 'curve fitting'. This is a question, not a criticism.
> 
> Phsst,
> 
> I guess that depends on what you call a small basket.  My "big
basket" has a
> gross of stocks in it.  I trade the top four dozen ("small basket"?) and
> always diversify among twenty issues.  Is this curve fitting?
> 
> Take care,
> 
> Steve
> ----- Original Message -----
> From: <phsst@xxxx>
> To: <amibroker@xxxxxxxxxxxxxxx>
> Sent: Wednesday, February 19, 2003 6:58 PM
> Subject: [amibroker] Re: System Testing Validity
> 
> 
> > <A well-designed system can trade dozens and dozens of issues with
> > eye-popping returns.  Technicians tend to "force" their approach
> > "favorite" issues (QQQ's, eminis, MSFT).  This is not logical.  You
> > can trade anything...why not trade orderly issues?>
> >
> > Is it not also true that:
> >
> > A poorly-designed system can trade dozens and dozens of issues with
> > eye-popping returns.
> >
> > I have always wanted my mech systems to work on the whole market of
> > securities. Isn't there any worry that by restricting a mech system to
> > a small basket of selected stocks that you might be doing some extreme
> > 'curve fitting'. This is a question, not a criticism.
> >
> > It seems that many here like to apply mech systems to a small subset
> > of securities. Is this approach really successful over the long haul,
> > or are you constantly 'stirring the pot' so to speak by fitting an
> > ever changing basket of stocks to the mech approach?
> >
> > Regards,
> >
> > Phsst
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "CedarCreekTrading" <kernish@xxxx>
> > wrote:
> > > >The other question I asked of no one in particular was ... When do
> > > you know your system has died ?  Even with valid thorough testing
> > > systems sometimes die and systems traders I think have to give some
> > > thought to want would constitute death to a system and then when the
> > > results for whatever reason approach that what they are going to do
> > > next.
> > >
> > > Fred,
> > >
> > > I've thought a lot about "death of a system" and I believe
> > technicians point
> > > to system failure with little regard to the change in supply and
demand
> > > and/or volatility (in the issue that they are trading).
> > Over-optimized and
> > > bad approaches to the markets will always fail (eventually).
> > >
> > > Sometimes, really great mechanical approaches can be compromised by
> > > "forcing" the system on issues that have random walk personalities.
> > > Well-behaved issues (trading vehicles that exhibit
> > accumulation/disturbution
> > > patterns) can return extrodinary profits for long periods of time.
> > Changes
> > > in supply and demand can cause a stock or commodity's behavior
to change
> > > drastically (and render repeated losses).
> > >
> > > Good, solid approaches will always work on a universe of
"well-behaved"
> > > stocks or commodities.  You can design objective criteria to pare
> > > ill-performing issues from any universe.  I think most of us can
> > conjure up
> > > a few rules about an issues's behavior and remove issues that don't
> > perform
> > > to specific standards.
> > >
> > > A well-designed system can trade dozens and dozens of issues with
> > > eye-popping returns.  Technicians tend to "force" their approach
> > "favorite"
> > > issues (QQQ's, eminis, MSFT).  This is not logical.  You can trade
> > > anything...why not trade orderly issues?
> > >
> > > That's my thoughts on "death".  Patterns change, but good systems
> > never die.
> > >
> > > Take care,
> > >
> > > Steve
> > >
> > >
> > > ----- Original Message -----
> > > From: <fctonetti@xxxx>
> > > To: <amibroker@xxxxxxxxxxxxxxx>
> > > Sent: Wednesday, February 19, 2003 1:32 PM
> > > Subject: [amibroker] Re: System Testing Validity
> > >
> > >
> > > > Steve,
> > > >
> > > > I think given the current environment we are in it is POSSIBLY no
> > > > longer absolutely necessary to be optimizing systems over that
long a
> > > > time frame.  However, pre 2000 IMHO one would have had to look
> > > > farther back in time then 1983 to see whether or not their
system was
> > > > viable and again IMHO at this juncture one needs to look
farther back
> > > > in time then the beginning of 2000 to see whether or not they
have a
> > > > viable system for exactly the same reason even though it's the
bear
> > > > that's familiar as opposed to the environment pre 2000.
> > > >
> > > > I am not predicting a bull market as beginning tomorrow at 3:30 or
> > > > that we've already turned the corner etc. only that clearly it
will
> > > > happen whether it's tomorrow, next week, month, year, decade or
> > > > whatever, I, like Herman says, could care less.  I only want
to know
> > > > that the systems I develop and trade work well in good markets and
> > > > bad.  If they do fine, if they don't ... NEXT !
> > > >
> > > > The other question I asked of no one in particular was ... When do
> > > > you know your system has died ?  Even with valid thorough testing
> > > > systems sometimes die and systems traders I think have to give
some
> > > > thought to want would constitute death to a system and then
when the
> > > > results for whatever reason approach that what they are going
to do
> > > > next.
> > > >
> > > > --- In amibroker@xxxxxxxxxxxxxxx, "Steve Davis" <sdavis@xxxx>
wrote:
> > > > > Fred,
> > > > >
> > > > > Please keep your insights coming. The subject of system
validity and
> > > > > robustness is certainly worthy of further discussion. In my
case, I
> > > > have a
> > > > > SP500 mechanical trading system optimized with historical
data from
> > > > 1983 to
> > > > > the present. My performance numbers during the optimzation
period
> > > > are
> > > > > CAR%=16%, MDD%=5.6%. Prior to the optimization period, the
profits
> > > > and
> > > > > drawdowns are both larger. Why am I using such a large
optimization
> > > > period?
> > > > > Because I am cautious and willing to sacrifice profits for lower
> > > > drawdowns.
> > > > > Would appreciate your thoughts on proper system validation
> > > > techniques. Am i
> > > > > being too cautious?
> > > > >
> > > > > My system performance is poor compared to your 100% CAR with
only
> > > > 3% MDD.
> > > > > Your system performance is far outside my reach right now.
> > > > >
> > > > > -Steve
> > > > >
> > > > > -----Original Message-----
> > > > > From: Fred <fctonetti@xxxx> [mailto:fctonetti@x...]
> > > > > Sent: Wednesday, February 19, 2003 2:44 PM
> > > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > > Subject: [amibroker] Re: The transcendental use of Data: An
> > > > application
> > > > >
> > > > >
> > > > > So be it ...
> > > > >
> > > > > The IDEA I was trying to share was validity of system
testing which
> > > > > includes out of sample results and why when this isn't done your
> > > > > likely to find out the hard way.  I find it of no particular
> > > > surprise
> > > > > that this has no particluar value to discretionary traders,
but then
> > > > > I also have no idea why discretionary traders need/want
tools like
> > > > AB.
> > > > >
> > > > > --- In amibroker@xxxxxxxxxxxxxxx, "Jayson" <jcasavant@xxxx>
wrote:
> > > > > > Fred,
> > > > > >
> > > > > > This is exactly what endears Dimitris to many of us here.
He has
> > > > > provided
> > > > > > you all the material needed to run the test your self. He has
> > > > > presented a
> > > > > > well thought out argument for why he believes it holds value.
> > > > There
> > > > > is no
> > > > > > hidden code, no magic box just the system as he
implemented it.
> > > > Run
> > > > > your own
> > > > > > tests and decide if the approach suites your own needs. If
after
> > > > > running the
> > > > > > system tests, optimized to your specifications and run
over the
> > > > > time frame
> > > > > > you deem necessary, you find his system holds no value
then simply
> > > > > move on.
> > > > > > If on the other hand you find some tid bit of it that holds
> > > > promise
> > > > > in your
> > > > > > eyes then run with it. Improve upon it and then, please,
post your
> > > > > > improvements. That is the goal of a group like this, to share
> > > > ideas
> > > > > and
> > > > > > hopefully grow from that sharing. Encouraging rather than
> > > > > discouraging this
> > > > > > type of sharing is what keeps active groups productive
else all we
> > > > > are doing
> > > > > > is offering free tech support for the product.
> > > > > >
> > > > > > Jayson
> > > > > > -----Original Message-----
> > > > > > From: Fred <fctonetti@xxxx> [mailto:fctonetti@x...]
> > > > > > Sent: Wednesday, February 19, 2003 2:18 PM
> > > > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > > > Subject: [amibroker] Re: The transcendental use of Data: An
> > > > > application
> > > > > >
> > > > > >
> > > > > > > Fred, read please my
> > > > > > > http://groups.yahoo.com/group/amibroker/message/34143
> > > > > >
> > > > > > Irrelevant ... You continue to very conveniently skirt the
issue.
> > > > > >
> > > > > > What timeframe was this optimization for ?
> > > > > >
> > > > > > Show the results of trading the same system using the same
> > > > > parameters
> > > > > > for some year long out of sample period prior to that.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >       Yahoo! Groups Sponsor
> > > > > >             ADVERTISEMENT
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx
> > > > > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > > > >
> > > > > > Check group FAQ at:
> > > > > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > > > >
> > > > > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> > > > Service.
> > > > >
> > > > >
> > > > > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > > > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > > >
> > > > > Check group FAQ at:
> > > > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > > >
> > > > > Your use of Yahoo! Groups is subject to
> > > > http://docs.yahoo.com/info/terms/
> > > >
> > > >
> > > > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > >
> > > > Check group FAQ at:
> > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > >
> > > > Your use of Yahoo! Groups is subject to
> > http://docs.yahoo.com/info/terms/
> > > >
> > > >
> >
> >
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> >
> > Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> > Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/
> >
> >


Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)

Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/