[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: On Robustness, Post #1



PureBytes Links

Trading Reference Links

Hi Dale,

Good question.  When someone posts something and I want to check it
out (I actually run at least the lite version on almost everything
posted here), I initially use their numbers.  If I want to explore
further I optimize (lol) the system on a different database, plot the
optimized parameters against performance measures and choose a set
of values that seems robust by eyeballing the graphs.  When I
wrote: "Test *unoptimized* system on small, mid & large cap stocks in
bull, bear & sideways market conditions, same parameters for all" I
was really trying to say, don't make a special case for each mkt cap
and mkt condition subtest by optimizing, use the same parameters for
all subtests.

Regards,

Mark

--- In amibroker@xxxxxxxxxxxxxxx, "dingo" <dingo@xxxx> wrote:
> Thanks for the post MF2!
>  
> Given Steve's example of the CMO5 which I assume is coded to detect
the
> cross of the indicator thru a value, how would you determine that
value
> for your intial testing?  This is the case below where you say "Test
> *unoptimized* system on small, mid & large cap stocks in bull, bear
&
> sideways market conditions, same parameters for all"
>  
> Thanks!
>  
> d
> 
> -----Original Message-----
> From: MarkF2 [mailto:feierstein@x...] 
> Sent: Saturday, November 01, 2003 2:50 PM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: [amibroker] On Robustness, Post #1
> 
> 
> This is in response to DT's and others' requests to provide more
> details on my 9 robustness criteria.
> 
> First some administrative anouncements, lol.  I've decided to
provide 
> them one-by-one, first due to my time constraints, second because I 
> feel that's the best way to discuss them and third because I want
to 
> see how this goes.  I welcome all constructive debate, especially 
> opposing views supported by quantitative analysis.  But if this 
> degenerates into a flame war, I've got better things to do with my 
> time.  Treat me with respect and I'll treat you with respect. 
There 
> seems to be a lot of interest in this topic, so let's please have a 
> collegial and productive discussion.  This is post 1 of 9 (not
> counting the dialog inbetween, let's see how far we can get :-).
> 
> Why care about robustness?  For whatever reasons, markets change. 
We 
> could spin our wheels forever discussing time series theory, serial 
> dependencies, random walk, nonstationarity, etc., like academicians
> do and get nowhere (as they do), or we can try to cut through the
crap
> and deal with it (the simple fact that markets constantly change). 
> My weapon of choice is robustness.  You could say I have a
robustness 
> obsession and my criteria are overkill.  But that's my choice and 
> you're free to make your own on how far you want to take this, if
at 
> all.
> 
> OK, I lied.  There will be some, very light discussion of
statistics 
> because some criteria are steeped in statistical theory.  But most
> can be reduced to simple, mechanical procedures that can be graphed
in
> a spreadsheet and visually and intuitively interpreted.  Others
> require simulation software and one requires proprietary software
but
> we'll cross that bridge when we come to it.  
> 
> Speaking of proprietary, there are some things I simply won't
> disclose, such as specific parameters for certain criteria.  So
please
> respect my wishes and don't ask.  I have my reasons.  So evaluate
this
> on your own and decide for yourself what place, if any, the criteria
> have in your trading.  They work great for me but I make no claim
that
> they're the Holy Grail of robustness and am sure that some of you
will
> come up with better ideas if there's enough interest and
discussion.  
> 
> With that long winded intro, here's Criterion #1:
> 
> Test *unoptimized* system on small, mid & large cap stocks in bull, 
> bear & sideways market conditions, same parameters for all.  I use
> the stocks of the S&P 600, 400, and 500 indices and 2 year bull,
bear
> and sideways periods (for a total of 6 years per stock).  Rationale
> behind this: to find systems that profitably *tested out in the
past*
> on a large number of (somewhat tradeable) stocks of varying market
> caps in multiple sectors under different market conditions, under
the 
> assumption that this indicates the system is robust enough to
> profitably *trade select issues in the future*.  More on robust
issue 
> selection in later criteria. Looking for net profitability on all
mkt 
> cap and mkt condition subtests, and profitable on the majority (>
> 50%) of issues in each subtest, the more the better.  Sometimes I
cut
> a system some slack if it's close on one or two subtests, it's a 
> judgement call.  My commission setting(s) in AB: proprietary, based
> on my *slippage* research using data from actual trades.  But you
> could choose an arbitrary say, 1% to get started.  Date settings for
> my 2 year intervals: proprietary but you can easily find your own
by 
> eyeballing a chart of a major index.  Just use the same ones each
> time so you compare apples to apples.   My lite version of this is 2
> year bull and bear periods on the ND100 and SP100 stocks, which I
> sometimes run as a quick pre-screen. Next time someone posts a
system,
> run it through the lite or full version.  Or test the systems in the
> AFL library.  The more systems you run through, the more intuitive
of
> a feel for robustness you'll get.  Note that I'm *not* saying you 
> shouldn't or can't successfully trade something that doesn't meet
> this standard, lol.  That's obviously not true!  I was asked to
> explain my robustness criteria and that's what I'm doing.  Period. 
> This criterion is a post-Amibroker creation, BTW.  Pre-Amibroker I
had
> a small test portfolio of diverse issues I used instead and it did a
> decent job. I run this now because I now (easily) can, *many* thanks
> to Tomasz.  If you're thinking, geez, why bother with this, ask
> yourself a simple question. *All else being equal*, would you feel
> more confident trading (with your money) a system that passes this
> test or one that fails it? 
> 
> Regards,
> 
> Mark
> 
> 
> 
> Yahoo! Groups Sponsor	
> 
> ADVERTISEMENT
>  
> <http://rd.yahoo.com/M=267637.4116730.5333196.12
61774/D=egroupweb/S=1705
> 632198:HM/A=1754452/R=0/SIG=11tn6fnpm/*http://ww
w.netflix.com/Default?mq
> so=60178324&partid=4116730> click here	
>  
> <http://us.adserver.yahoo.com/l?M=267637.4116730
.5333196.1261774/D=egrou
> pmail/S=:HM/A=1754452/rand=847508790> 	
> 
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html 
> 
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service
> <http://docs.yahoo.com/info/terms/> .


------------------------ Yahoo! Groups Sponsor ---------------------~-->
Rent DVDs from home.
Over 14,500 titles. Free Shipping
& No Late Fees. Try Netflix for FREE!
http://us.click.yahoo.com/I3w.vC/hP.FAA/3jkFAA/GHeqlB/TM
---------------------------------------------------------------------~->

Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/