[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: System Testing Validity



PureBytes Links

Trading Reference Links

<A well-designed system can trade dozens and dozens of issues with
eye-popping returns.  Technicians tend to "force" their approach
"favorite" issues (QQQ's, eminis, MSFT).  This is not logical.  You
can trade anything...why not trade orderly issues?>

Is it not also true that:

A poorly-designed system can trade dozens and dozens of issues with
eye-popping returns.

I have always wanted my mech systems to work on the whole market of
securities. Isn't there any worry that by restricting a mech system to
a small basket of selected stocks that you might be doing some extreme
'curve fitting'. This is a question, not a criticism.

It seems that many here like to apply mech systems to a small subset
of securities. Is this approach really successful over the long haul,
or are you constantly 'stirring the pot' so to speak by fitting an
ever changing basket of stocks to the mech approach?

Regards,

Phsst 

--- In amibroker@xxxxxxxxxxxxxxx, "CedarCreekTrading" <kernish@xxxx>
wrote:
> >The other question I asked of no one in particular was ... When do
> you know your system has died ?  Even with valid thorough testing
> systems sometimes die and systems traders I think have to give some
> thought to want would constitute death to a system and then when the
> results for whatever reason approach that what they are going to do
> next.
> 
> Fred,
> 
> I've thought a lot about "death of a system" and I believe
technicians point
> to system failure with little regard to the change in supply and demand
> and/or volatility (in the issue that they are trading). 
Over-optimized and
> bad approaches to the markets will always fail (eventually).
> 
> Sometimes, really great mechanical approaches can be compromised by
> "forcing" the system on issues that have random walk personalities.
> Well-behaved issues (trading vehicles that exhibit
accumulation/disturbution
> patterns) can return extrodinary profits for long periods of time. 
Changes
> in supply and demand can cause a stock or commodity's behavior to change
> drastically (and render repeated losses).
> 
> Good, solid approaches will always work on a universe of "well-behaved"
> stocks or commodities.  You can design objective criteria to pare
> ill-performing issues from any universe.  I think most of us can
conjure up
> a few rules about an issues's behavior and remove issues that don't
perform
> to specific standards.
> 
> A well-designed system can trade dozens and dozens of issues with
> eye-popping returns.  Technicians tend to "force" their approach
"favorite"
> issues (QQQ's, eminis, MSFT).  This is not logical.  You can trade
> anything...why not trade orderly issues?
> 
> That's my thoughts on "death".  Patterns change, but good systems
never die.
> 
> Take care,
> 
> Steve
> 
> 
> ----- Original Message -----
> From: <fctonetti@xxxx>
> To: <amibroker@xxxxxxxxxxxxxxx>
> Sent: Wednesday, February 19, 2003 1:32 PM
> Subject: [amibroker] Re: System Testing Validity
> 
> 
> > Steve,
> >
> > I think given the current environment we are in it is POSSIBLY no
> > longer absolutely necessary to be optimizing systems over that long a
> > time frame.  However, pre 2000 IMHO one would have had to look
> > farther back in time then 1983 to see whether or not their system was
> > viable and again IMHO at this juncture one needs to look farther back
> > in time then the beginning of 2000 to see whether or not they have a
> > viable system for exactly the same reason even though it's the bear
> > that's familiar as opposed to the environment pre 2000.
> >
> > I am not predicting a bull market as beginning tomorrow at 3:30 or
> > that we've already turned the corner etc. only that clearly it will
> > happen whether it's tomorrow, next week, month, year, decade or
> > whatever, I, like Herman says, could care less.  I only want to know
> > that the systems I develop and trade work well in good markets and
> > bad.  If they do fine, if they don't ... NEXT !
> >
> > The other question I asked of no one in particular was ... When do
> > you know your system has died ?  Even with valid thorough testing
> > systems sometimes die and systems traders I think have to give some
> > thought to want would constitute death to a system and then when the
> > results for whatever reason approach that what they are going to do
> > next.
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "Steve Davis" <sdavis@xxxx> wrote:
> > > Fred,
> > >
> > > Please keep your insights coming. The subject of system validity and
> > > robustness is certainly worthy of further discussion. In my case, I
> > have a
> > > SP500 mechanical trading system optimized with historical data from
> > 1983 to
> > > the present. My performance numbers during the optimzation period
> > are
> > > CAR%=16%, MDD%=5.6%. Prior to the optimization period, the profits
> > and
> > > drawdowns are both larger. Why am I using such a large optimization
> > period?
> > > Because I am cautious and willing to sacrifice profits for lower
> > drawdowns.
> > > Would appreciate your thoughts on proper system validation
> > techniques. Am i
> > > being too cautious?
> > >
> > > My system performance is poor compared to your 100% CAR with only
> > 3% MDD.
> > > Your system performance is far outside my reach right now.
> > >
> > > -Steve
> > >
> > > -----Original Message-----
> > > From: Fred <fctonetti@xxxx> [mailto:fctonetti@x...]
> > > Sent: Wednesday, February 19, 2003 2:44 PM
> > > To: amibroker@xxxxxxxxxxxxxxx
> > > Subject: [amibroker] Re: The transcendental use of Data: An
> > application
> > >
> > >
> > > So be it ...
> > >
> > > The IDEA I was trying to share was validity of system testing which
> > > includes out of sample results and why when this isn't done your
> > > likely to find out the hard way.  I find it of no particular
> > surprise
> > > that this has no particluar value to discretionary traders, but then
> > > I also have no idea why discretionary traders need/want tools like
> > AB.
> > >
> > > --- In amibroker@xxxxxxxxxxxxxxx, "Jayson" <jcasavant@xxxx> wrote:
> > > > Fred,
> > > >
> > > > This is exactly what endears Dimitris to many of us here. He has
> > > provided
> > > > you all the material needed to run the test your self. He has
> > > presented a
> > > > well thought out argument for why he believes it holds value.
> > There
> > > is no
> > > > hidden code, no magic box just the system as he implemented it.
> > Run
> > > your own
> > > > tests and decide if the approach suites your own needs. If after
> > > running the
> > > > system tests, optimized to your specifications and run over the
> > > time frame
> > > > you deem necessary, you find his system holds no value then simply
> > > move on.
> > > > If on the other hand you find some tid bit of it that holds
> > promise
> > > in your
> > > > eyes then run with it. Improve upon it and then, please, post your
> > > > improvements. That is the goal of a group like this, to share
> > ideas
> > > and
> > > > hopefully grow from that sharing. Encouraging rather than
> > > discouraging this
> > > > type of sharing is what keeps active groups productive else all we
> > > are doing
> > > > is offering free tech support for the product.
> > > >
> > > > Jayson
> > > > -----Original Message-----
> > > > From: Fred <fctonetti@xxxx> [mailto:fctonetti@x...]
> > > > Sent: Wednesday, February 19, 2003 2:18 PM
> > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > Subject: [amibroker] Re: The transcendental use of Data: An
> > > application
> > > >
> > > >
> > > > > Fred, read please my
> > > > > http://groups.yahoo.com/group/amibroker/message/34143
> > > >
> > > > Irrelevant ... You continue to very conveniently skirt the issue.
> > > >
> > > > What timeframe was this optimization for ?
> > > >
> > > > Show the results of trading the same system using the same
> > > parameters
> > > > for some year long out of sample period prior to that.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >       Yahoo! Groups Sponsor
> > > >             ADVERTISEMENT
> > > >
> > > >
> > > >
> > > >
> > > > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > >
> > > > Check group FAQ at:
> > > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > >
> > > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> > Service.
> > >
> > >
> > > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > >
> > > Check group FAQ at:
> > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > >
> > > Your use of Yahoo! Groups is subject to
> > http://docs.yahoo.com/info/terms/
> >
> >
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> >
> > Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> > Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/
> >
> >


Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)

Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/