[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: System Testing Validity



PureBytes Links

Trading Reference Links

Steve,

I think given the current environment we are in it is POSSIBLY no 
longer absolutely necessary to be optimizing systems over that long a 
time frame.  However, pre 2000 IMHO one would have had to look 
farther back in time then 1983 to see whether or not their system was 
viable and again IMHO at this juncture one needs to look farther back 
in time then the beginning of 2000 to see whether or not they have a 
viable system for exactly the same reason even though it's the bear 
that's familiar as opposed to the environment pre 2000.

I am not predicting a bull market as beginning tomorrow at 3:30 or 
that we've already turned the corner etc. only that clearly it will 
happen whether it's tomorrow, next week, month, year, decade or 
whatever, I, like Herman says, could care less.  I only want to know 
that the systems I develop and trade work well in good markets and 
bad.  If they do fine, if they don't ... NEXT !

The other question I asked of no one in particular was ... When do 
you know your system has died ?  Even with valid thorough testing 
systems sometimes die and systems traders I think have to give some 
thought to want would constitute death to a system and then when the 
results for whatever reason approach that what they are going to do 
next.

--- In amibroker@xxxxxxxxxxxxxxx, "Steve Davis" <sdavis@xxxx> wrote:
> Fred,
> 
> Please keep your insights coming. The subject of system validity and
> robustness is certainly worthy of further discussion. In my case, I 
have a
> SP500 mechanical trading system optimized with historical data from 
1983 to
> the present. My performance numbers during the optimzation period 
are
> CAR%=16%, MDD%=5.6%. Prior to the optimization period, the profits 
and
> drawdowns are both larger. Why am I using such a large optimization 
period?
> Because I am cautious and willing to sacrifice profits for lower 
drawdowns.
> Would appreciate your thoughts on proper system validation 
techniques. Am i
> being too cautious?
> 
> My system performance is poor compared to your 100% CAR with only 
3% MDD.
> Your system performance is far outside my reach right now.
> 
> -Steve
> 
> -----Original Message-----
> From: Fred <fctonetti@xxxx> [mailto:fctonetti@x...]
> Sent: Wednesday, February 19, 2003 2:44 PM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: [amibroker] Re: The transcendental use of Data: An 
application
> 
> 
> So be it ...
> 
> The IDEA I was trying to share was validity of system testing which
> includes out of sample results and why when this isn't done your
> likely to find out the hard way.  I find it of no particular 
surprise
> that this has no particluar value to discretionary traders, but then
> I also have no idea why discretionary traders need/want tools like 
AB.
> 
> --- In amibroker@xxxxxxxxxxxxxxx, "Jayson" <jcasavant@xxxx> wrote:
> > Fred,
> >
> > This is exactly what endears Dimitris to many of us here. He has
> provided
> > you all the material needed to run the test your self. He has
> presented a
> > well thought out argument for why he believes it holds value. 
There
> is no
> > hidden code, no magic box just the system as he implemented it. 
Run
> your own
> > tests and decide if the approach suites your own needs. If after
> running the
> > system tests, optimized to your specifications and run over the
> time frame
> > you deem necessary, you find his system holds no value then simply
> move on.
> > If on the other hand you find some tid bit of it that holds 
promise
> in your
> > eyes then run with it. Improve upon it and then, please, post your
> > improvements. That is the goal of a group like this, to share 
ideas
> and
> > hopefully grow from that sharing. Encouraging rather than
> discouraging this
> > type of sharing is what keeps active groups productive else all we
> are doing
> > is offering free tech support for the product.
> >
> > Jayson
> > -----Original Message-----
> > From: Fred <fctonetti@xxxx> [mailto:fctonetti@x...]
> > Sent: Wednesday, February 19, 2003 2:18 PM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: [amibroker] Re: The transcendental use of Data: An
> application
> >
> >
> > > Fred, read please my
> > > http://groups.yahoo.com/group/amibroker/message/34143
> >
> > Irrelevant ... You continue to very conveniently skirt the issue.
> >
> > What timeframe was this optimization for ?
> >
> > Show the results of trading the same system using the same
> parameters
> > for some year long out of sample period prior to that.
> >
> >
> >
> >
> >
> >
> >
> >       Yahoo! Groups Sponsor
> >             ADVERTISEMENT
> >
> >
> >
> >
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> >
> > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> > Your use of Yahoo! Groups is subject to the Yahoo! Terms of 
Service.
> 
> 
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> 
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> 
> Your use of Yahoo! Groups is subject to 
http://docs.yahoo.com/info/terms/


Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)

Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/