[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: Robustness Criteria, Condensed 1-5. was Robustness, Post #1



PureBytes Links

Trading Reference Links

the bootstrap

--- In amibroker@xxxxxxxxxxxxxxx, "Fred" <fctonetti@xxxx> wrote:
> Mark,
> 
> In particular what you have in mind by "simulation" ...
> 
> --- In amibroker@xxxxxxxxxxxxxxx, "Fred" <fctonetti@xxxx> wrote:
> > Mark,
> > 
> > Can you add a few more words behind #4 & 5 ?
> > 
> > --- In amibroker@xxxxxxxxxxxxxxx, "quanttrader714" 
> > <quanttrader714@xxxx> wrote:
> > > Robustness Criteria, Condensed 1-5.
> > > 
> > > 1. Test on small, mid & large cap stocks in bull, bear & sideways
> > > markets.  
> > > 2. Evaluate performance on top 20% most actively traded small, 
> mid &
> > > large cap stocks.
> > > 3. Graph and evaluate system performance consistency (%
> profit/trade
> > > and % profit/bar) on select stocks.
> > > 4. Perform simulation to estimate probability of profit in 10 
> trades
> > > (for select stocks).
> > > 5. Perform simulation to estimate future drawdown (for select 
> > stocks).
> > > 
> > > --- In amibroker@xxxxxxxxxxxxxxx, "Fred" <fctonetti@xxxx> wrote:
> > > > At the moment I'd settle for having a one line description of 
> #2 -
> > > #8.
> > > > 
> > > > --- In amibroker@xxxxxxxxxxxxxxx, "leonardot19" 
> > > <leo.timmermans@xxxx> 
> > > > wrote:
> > > > > Hi Anthony, Mark,
> > > > > 
> > > > > This is a good idea. This will allow for the less gifted, 
> like 
> > > > myself 
> > > > > (lol) to follow more closely.
> > > > > 
> > > > > Thanks
> > > > > Leo
> > > > > 
> > > > > 
> > > > > --- In amibroker@xxxxxxxxxxxxxxx, "Anthony Faragasso" 
> > > > <ajf1111@xxxx> 
> > > > > wrote:
> > > > > > Mark,
> > > > > > 
> > > > > > Thanks for the thread....How about exploring each of your 9 
> > > > points 
> > > > > of
> > > > > > Robustness with a sample simple System....then (you / we ) 
> > can 
> > > > > apply each
> > > > > > point to this sample system....with your direction....and
> > > discuss 
> > > > > why this
> > > > > > system would be accepted or not as pertains to the specific 
> > > > point...
> > > > > > 
> > > > > > Anthony
> > > > > > 
> > > > > > 
> > > > > > ----- Original Message ----- 
> > > > > > From: "quanttrader714" <quanttrader714@xxxx>
> > > > > > To: <amibroker@xxxxxxxxxxxxxxx>
> > > > > > Sent: Sunday, November 02, 2003 11:13 AM
> > > > > > Subject: [amibroker] Re: On Robustness, Post #1
> > > > > > 
> > > > > > 
> > > > > > > Hi Dale,
> > > > > > >
> > > > > > > Good question.  When someone posts something and I want 
> to 
> > > > check 
> > > > > it
> > > > > > > out (I actually run at least the lite version on almost 
> > > > everything
> > > > > > > posted here), I initially use their numbers.  If I want 
> to 
> > > > explore
> > > > > > > further I optimize (lol) the system on a different 
> > database, 
> > > > plot 
> > > > > the
> > > > > > > optimized parameters against performance measures and 
> choose
> > > a 
> > > > set
> > > > > > > of values that seems robust by eyeballing the graphs.  
> When 
> > I
> > > > > > > wrote: "Test *unoptimized* system on small, mid & large 
> cap 
> > > > > stocks in
> > > > > > > bull, bear & sideways market conditions, same parameters 
> > for 
> > > > all" 
> > > > > I
> > > > > > > was really trying to say, don't make a special case for 
> > each 
> > > > mkt 
> > > > > cap
> > > > > > > and mkt condition subtest by optimizing, use the same 
> > > > parameters 
> > > > > for
> > > > > > > all subtests.
> > > > > > >
> > > > > > > Regards,
> > > > > > >
> > > > > > > Mark
> > > > > > >
> > > > > > > --- In amibroker@xxxxxxxxxxxxxxx, "dingo" <dingo@xxxx> 
> > wrote:
> > > > > > > > Thanks for the post MF2!
> > > > > > > >
> > > > > > > > Given Steve's example of the CMO5 which I assume is 
> coded
> > > to 
> > > > > detect
> > > > > > > the
> > > > > > > > cross of the indicator thru a value, how would you
> > > determine 
> > > > > that
> > > > > > > value
> > > > > > > > for your intial testing?  This is the case below where 
> > you 
> > > > > say "Test
> > > > > > > > *unoptimized* system on small, mid & large cap stocks 
> in 
> > > > bull, 
> > > > > bear
> > > > > > > &
> > > > > > > > sideways market conditions, same parameters for all"
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > d
> > > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: MarkF2 [mailto:feierstein@x...]
> > > > > > > > Sent: Saturday, November 01, 2003 2:50 PM
> > > > > > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > > > > > Subject: [amibroker] On Robustness, Post #1
> > > > > > > >
> > > > > > > >
> > > > > > > > This is in response to DT's and others' requests to 
> > provide 
> > > > more
> > > > > > > > details on my 9 robustness criteria.
> > > > > > > >
> > > > > > > > First some administrative anouncements, lol.  I've 
> decided
> > > to
> > > > > > > provide
> > > > > > > > them one-by-one, first due to my time constraints, 
> second 
> > > > > because I
> > > > > > > > feel that's the best way to discuss them and third 
> because
> > > I 
> > > > > want
> > > > > > > to
> > > > > > > > see how this goes.  I welcome all constructive debate, 
> > > > > especially
> > > > > > > > opposing views supported by quantitative analysis.  But 
> > if 
> > > > this
> > > > > > > > degenerates into a flame war, I've got better things to 
> > do 
> > > > with 
> > > > > my
> > > > > > > > time.  Treat me with respect and I'll treat you with
> > > respect.
> > > > > > > There
> > > > > > > > seems to be a lot of interest in this topic, so let's
> > > please 
> > > > > have a
> > > > > > > > collegial and productive discussion.  This is post 1 of 
> 9
> > > (not
> > > > > > > > counting the dialog inbetween, let's see how far we can 
> > get
> > > :-
> > > > ).
> > > > > > > >
> > > > > > > > Why care about robustness?  For whatever reasons, 
> markets 
> > > > > change.
> > > > > > > We
> > > > > > > > could spin our wheels forever discussing time series
> > > theory, 
> > > > > serial
> > > > > > > > dependencies, random walk, nonstationarity, etc., like 
> > > > > academicians
> > > > > > > > do and get nowhere (as they do), or we can try to cut
> > > through 
> > > > > the
> > > > > > > crap
> > > > > > > > and deal with it (the simple fact that markets 
> constantly 
> > > > > change).
> > > > > > > > My weapon of choice is robustness.  You could say I 
> have a
> > > > > > > robustness
> > > > > > > > obsession and my criteria are overkill.  But that's my
> > > choice 
> > > > > and
> > > > > > > > you're free to make your own on how far you want to 
> take 
> > > > this, 
> > > > > if
> > > > > > > at
> > > > > > > > all.
> > > > > > > >
> > > > > > > > OK, I lied.  There will be some, very light discussion 
> of
> > > > > > > statistics
> > > > > > > > because some criteria are steeped in statistical 
> theory. 
> > > But 
> > > > > most
> > > > > > > > can be reduced to simple, mechanical procedures that 
> can 
> > be 
> > > > > graphed
> > > > > > > in
> > > > > > > > a spreadsheet and visually and intuitively 
> interpreted.  
> > > > Others
> > > > > > > > require simulation software and one requires 
> proprietary 
> > > > > software
> > > > > > > but
> > > > > > > > we'll cross that bridge when we come to it.
> > > > > > > >
> > > > > > > > Speaking of proprietary, there are some things I simply
> > > won't
> > > > > > > > disclose, such as specific parameters for certain 
> > criteria.
> > >  
> > > > So
> > > > > > > please
> > > > > > > > respect my wishes and don't ask.  I have my reasons.  
> So 
> > > > > evaluate
> > > > > > > this
> > > > > > > > on your own and decide for yourself what place, if any, 
> > the 
> > > > > criteria
> > > > > > > > have in your trading.  They work great for me but I 
> make 
> > no 
> > > > > claim
> > > > > > > that
> > > > > > > > they're the Holy Grail of robustness and am sure that 
> some
> > > of 
> > > > > you
> > > > > > > will
> > > > > > > > come up with better ideas if there's enough interest and
> > > > > > > discussion.
> > > > > > > >
> > > > > > > > With that long winded intro, here's Criterion #1:
> > > > > > > >
> > > > > > > > Test *unoptimized* system on small, mid & large cap 
> stocks
> > > in 
> > > > > bull,
> > > > > > > > bear & sideways market conditions, same parameters for 
> > all.
> > >  
> > > > I 
> > > > > use
> > > > > > > > the stocks of the S&P 600, 400, and 500 indices and 2 
> > year 
> > > > bull,
> > > > > > > bear
> > > > > > > > and sideways periods (for a total of 6 years per 
> stock).  
> > > > > Rationale
> > > > > > > > behind this: to find systems that profitably *tested 
> out 
> > in 
> > > > the
> > > > > > > past*
> > > > > > > > on a large number of (somewhat tradeable) stocks of 
> > varying 
> > > > > market
> > > > > > > > caps in multiple sectors under different market 
> > conditions, 
> > > > > under
> > > > > > > the
> > > > > > > > assumption that this indicates the system is robust 
> enough
> > > to
> > > > > > > > profitably *trade select issues in the future*.  More 
> on 
> > > > robust
> > > > > > > issue
> > > > > > > > selection in later criteria. Looking for net 
> profitability
> > > on 
> > > > > all
> > > > > > > mkt
> > > > > > > > cap and mkt condition subtests, and profitable on the 
> > > > majority 
> > > > > (>
> > > > > > > > 50%) of issues in each subtest, the more the better.  
> > > > Sometimes 
> > > > > I
> > > > > > > cut
> > > > > > > > a system some slack if it's close on one or two 
> subtests, 
> > > > it's a
> > > > > > > > judgement call.  My commission setting(s) in AB:
> > > proprietary, 
> > > > > based
> > > > > > > > on my *slippage* research using data from actual 
> trades. 
> > > But 
> > > > > you
> > > > > > > > could choose an arbitrary say, 1% to get started.  Date 
> > > > > settings for
> > > > > > > > my 2 year intervals: proprietary but you can easily find
> > > your 
> > > > > own
> > > > > > > by
> > > > > > > > eyeballing a chart of a major index.  Just use the same
> > > ones 
> > > > > each
> > > > > > > > time so you compare apples to apples.   My lite version 
> > of 
> > > > this 
> > > > > is 2
> > > > > > > > year bull and bear periods on the ND100 and SP100 
> stocks, 
> > > > which 
> > > > > I
> > > > > > > > sometimes run as a quick pre-screen. Next time someone
> > > posts a
> > > > > > > system,
> > > > > > > > run it through the lite or full version.  Or test the
> > > systems 
> > > > > in the
> > > > > > > > AFL library.  The more systems you run through, the 
> more 
> > > > > intuitive
> > > > > > > of
> > > > > > > > a feel for robustness you'll get.  Note that I'm *not*
> > > saying 
> > > > > you
> > > > > > > > shouldn't or can't successfully trade something that
> > > doesn't 
> > > > > meet
> > > > > > > > this standard, lol.  That's obviously not true!  I was
> > > asked 
> > > > to
> > > > > > > > explain my robustness criteria and that's what I'm 
> > doing.  
> > > > > Period.
> > > > > > > > This criterion is a post-Amibroker creation, BTW.  Pre-
> > > > > Amibroker I
> > > > > > > had
> > > > > > > > a small test portfolio of diverse issues I used instead 
> > and 
> > > > it 
> > > > > did a
> > > > > > > > decent job. I run this now because I now (easily) can,
> > > *many* 
> > > > > thanks
> > > > > > > > to Tomasz.  If you're thinking, geez, why bother with 
> > this, 
> > > > ask
> > > > > > > > yourself a simple question. *All else being equal*, 
> would
> > > you 
> > > > > feel
> > > > > > > > more confident trading (with your money) a system that
> > > passes 
> > > > > this
> > > > > > > > test or one that fails it?
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > >
> > > > > > > > Mark
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Yahoo! Groups Sponsor
> > > > > > > >
> > > > > > > > ADVERTISEMENT
> > > > > > > >
> > > > > > > > <http://rd.yahoo.com/M=267637.4116730.5333196.12
> > > > > > > 61774/D=egroupweb/S=1705
> > > > > > > > 632198:HM/A=1754452/R=0/SIG=11tn6fnpm/*http://ww
> > > > > > > w.netflix.com/Default?mq
> > > > > > > > so=60178324&partid=4116730> click here
> > > > > > > >
> > > > > > > > <http://us.adserver.yahoo.com/l?M=267637.4116730
> > > > > > > .5333196.1261774/D=egrou
> > > > > > > > pmail/S=:HM/A=1754452/rand=847508790>
> > > > > > > >
> > > > > > > > Send BUG REPORTS to bugs@xxxx
> > > > > > > > Send SUGGESTIONS to suggest@xxxx
> > > > > > > > -----------------------------------------
> > > > > > > > Post AmiQuote-related messages ONLY to: 
> > > > amiquote@xxxxxxxxxxxxxxx
> > > > > > > > (Web page: 
> > http://groups.yahoo.com/group/amiquote/messages/)
> > > > > > > > --------------------------------------------
> > > > > > > > Check group FAQ at:
> > > > > > > > 
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > > > > > >
> > > > > > > > Your use of Yahoo! Groups is subject to the Yahoo! 
> Terms 
> > of 
> > > > > Service
> > > > > > > > <http://docs.yahoo.com/info/terms/> .
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Send BUG REPORTS to bugs@xxxx
> > > > > > > Send SUGGESTIONS to suggest@xxxx
> > > > > > > -----------------------------------------
> > > > > > > Post AmiQuote-related messages ONLY to: am
> > > iquote@xxxxxxxxxxxxxxx
> > > > > > > (Web page: 
> http://groups.yahoo.com/group/amiquote/messages/)
> > > > > > > --------------------------------------------
> > > > > > > Check group FAQ at:
> > > > > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > > > > >
> > > > > > > Your use of Yahoo! Groups is subject to 
> > > > > http://docs.yahoo.com/info/terms/
> > > > > > >
> > > > > > >
> > > > > > 
> > > > > > 
> > > > > > ---
> > > > > > Outgoing mail is certified Virus Free.
> > > > > > Checked by AVG anti-virus system (http://www.grisoft.com).
> > > > > > Version: 6.0.535 / Virus Database: 330 - Release Date: 
> > 11/1/2003


------------------------ Yahoo! Groups Sponsor ---------------------~-->
Rent DVDs Online - Over 14,500 titles.
No Late Fees & Free Shipping.
Try Netflix for FREE!
http://us.click.yahoo.com/xlw.sC/XP.FAA/3jkFAA/GHeqlB/TM
---------------------------------------------------------------------~->

Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx 
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/