[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: System Testing Validity



PureBytes Links

Trading Reference Links

Al,

Herman and I have had some discussion about trading the equity curve 
and we are of differing opinions.  My personal belief is that if one  
alternates systems based on which has log of equity above the linear 
regression of the log of equity that you will be catching those 
systems at their peak just prior to them "wanting" to return to the 
mean i.e. one just missed the best time.

My perspective on when systems die is really very simple ... if CAR 
falls off the table or DD's exceed historical maximums it's at least 
time to start asking questions.

As far as "Good Systems Never Die" goes ... the problem here is that 
of course one is never sure whether one has a good system until one 
trades it real time for awhile and even then it has the distinct 
possibility of going belly up any way, ergo the reason to test, test, 
test and see what the theoretical real time results are by looking at 
the out of sample etc.  The longer the optimization/test period and 
out of sample are the more likely one is to have found a workable 
solution.

Fred

--- In amibroker@xxxxxxxxxxxxxxx, "Al Venosa" <avcinci@xxxx> wrote:
> Fred,
> 
> I don't know when a system 'dies'. But, as an inexperienced, 
aspiring 
> trader, might I suggest examining and keeping close watch of your 
equity 
> curve? If it begins to 'die', perhaps it's either time to abandon 
it or 
> re-optimize your parameter values. There have been several posts on 
this 
> board about using the equity curve as a way of modifying your 
entries 
> (equity feedback). In fact, the January issue of Active Trader 
magazine had 
> an article in it where the author suggested keeping a moving 
average of your 
> equity curve, and stop trading your system when the equity fell 
below the ma 
> and resume trading it again when it crossed back above the ma. This 
requires 
> you to keep track of non-existent trades to keep your equity curve 
active, 
> but the approach sounds logical to me. In other words, only trade 
when the 
> equity curve is on or above its ma. This is but one of perhaps many 
> different ways to use equity feedback. Have you or anyone else 
played with 
> this approach much?
> 
> Al Venosa
> 
> >From: "Fred " Reply-To: amibroker@xxxxxxxxxxxxxxx To: 
> >amibroker@xxxxxxxxxxxxxxx Subject: [amibroker] Re: System Testing 
Validity 
> >Date: Wed, 19 Feb 2003 20:32:52 -0000
> >
> >Steve,
> >
> >I think given the current environment we are in it is POSSIBLY no 
longer 
> >absolutely necessary to be optimizing systems over that long a 
time frame. 
> >However, pre 2000 IMHO one would have had to look farther back in 
time then 
> >1983 to see whether or not their system was viable and again IMHO 
at this 
> >juncture one needs to look farther back in time then the beginning 
of 2000 
> >to see whether or not they have a viable system for exactly the 
same reason 
> >even though it's the bear that's familiar as opposed to the 
environment pre 
> >2000.
> >
> >I am not predicting a bull market as beginning tomorrow at 3:30 or 
that 
> >we've already turned the corner etc. only that clearly it will 
happen 
> >whether it's tomorrow, next week, month, year, decade or whatever, 
I, like 
> >Herman says, could care less. I only want to know that the systems 
I 
> >develop and trade work well in good markets and bad. If they do 
fine, if 
> >they don't ... NEXT !
> >
> >The other question I asked of no one in particular was ... When do 
you know 
> >your system has died ? Even with valid thorough testing systems 
sometimes 
> >die and systems traders I think have to give some thought to want 
would 
> >constitute death to a system and then when the results for 
whatever reason 
> >approach that what they are going to do next.
> >
> >--- In amibroker@xxxxxxxxxxxxxxx, "Steve Davis" wrote: > Fred, > > 
Please 
> >keep your insights coming. The subject of system validity and > 
robustness 
> >is certainly worthy of further discussion. In my case, I have a > 
SP500 
> >mechanical trading system optimized with historical data from 1983 
to > the 
> >present. My performance numbers during the optimzation period are 
> 
> >CAR%=16%, MDD%=5.6%. Prior to the optimization period, the profits 
and > 
> >drawdowns are both larger. Why am I using such a large 
optimization period? 
> > > Because I am cautious and willing to sacrifice profits for 
lower 
> >drawdowns. > Would appreciate your thoughts on proper system 
validation 
> >techniques. Am i > being too cautious? > > My system performance 
is poor 
> >compared to your 100% CAR with only 3% MDD. > Your system 
performance is 
> >far outside my reach right now. > > -Steve > > -----Original 
Message----- > 
> >From: Fred [mailto:fctonetti@x...] > Sent: Wednesday, February 19, 
2003 
> >2:44 PM > To: amibroker@xxxxxxxxxxxxxxx > Subject: [amibroker] Re: 
The 
> >transcendental use of Data: An application > > > So be it ... > > 
The IDEA 
> >I was trying to share was validity of system testing which > 
includes out 
> >of sample results and why when this isn't done your > likely to 
find out 
> >the hard way. I find it of no particular surprise > that this has 
no 
> >particluar value to discretionary traders, but then > I also have 
no idea 
> >why discretionary traders need/want tools like AB. > > --- In 
> >amibroker@xxxxxxxxxxxxxxx, "Jayson" wrote: > > Fred, > > > > This 
is 
> >exactly what endears Dimitris to many of us here. He has > 
provided > > you 
> >all the material needed to run the test your self. He has > 
presented a > > 
> >well thought out argument for why he believes it holds value. 
There > is no 
> > > > hidden code, no magic box just the system as he implemented 
it. Run > 
> >your own > > tests and decide if the approach suites your own 
needs. If 
> >after > running the > > system tests, optimized to your 
specifications and 
> >run over the > time frame > > you deem necessary, you find his 
system holds 
> >no value then simply > move on. > > If on the other hand you find 
some tid 
> >bit of it that holds promise > in your > > eyes then run with it. 
Improve 
> >upon it and then, please, post your > > improvements. That is the 
goal of a 
> >group like this, to share ideas > and > > hopefully grow from that 
sharing. 
> >Encouraging rather than > discouraging this > > type of sharing is 
what 
> >keeps active groups productive else all we > are doing > > is 
offering free 
> >tech support for the product. > > > > Jayson > > -----Original 
Message----- 
> > > > From: Fred [mailto:fctonetti@x...] > > Sent: Wednesday, 
February 19, 
> >2003 2:18 PM > > To: amibroker@xxxxxxxxxxxxxxx > > Subject: 
[amibroker] Re: 
> >The transcendental use of Data: An > application > > > > > > > 
Fred, read 
> >please my > > > 
http://groups.yahoo.com/group/amibroker/message/34143 > > > 
> > > Irrelevant ... You continue to very conveniently skirt the 
issue. > > > 
> > > What timeframe was this optimization for ? > > > > Show the 
results of 
> >trading the same system using the same > parameters > > for some 
year long 
> >out of sample period prior to that. > > > > > > > > > > > > > > > 
> Yahoo! 
> >Groups Sponsor > > ADVERTISEMENT > > > > > > > > > > Post AmiQuote-
related 
> >messages ONLY to: amiquote@xxxxxxxxxxxxxxx > > (Web page: 
> >http://groups.yahoo.com/group/amiquote/messages/) > > > > Check 
group FAQ 
> >at: > > 
http://groups.yahoo.com/group/amibroker/files/groupfaq.html > > > > 
> >Your use of Yahoo! Groups is subject to the Yahoo! Terms of 
Service. > > > 
> >Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx > 
(Web 
> >page: http://groups.yahoo.com/group/amiquote/messages/) > > Check 
group FAQ 
> >at: > http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
> > Your 
> >use of Yahoo! Groups is subject to 
http://docs.yahoo.com/info/terms/
> >
> 
> _________________________________________________________________
> STOP MORE SPAM with the new MSN 8 and get 2 months FREE*  
> http://join.msn.com/?page=features/junkmail


Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)

Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/