|
PureBytes Links
Trading Reference Links
|
For the robustness stuff I posted and many other methodologies, I'd
agree. But with all due respect to Gary, what made me want to
respond was that if one didn't already know the info in Howard's post
and if one has little if any experience with MCS there are lots of
pitfalls, especially with trying to bridge the gap between expected
and actual realtime results. And trying to do simulations without
being aware of them and ways to mitigate gives poor results and a bad
name to a perfectly valid and useful process.
--- In amibroker@xxxxxxxxxxxxxxx, "Fred" <fctonetti@xxxx> wrote:
> Mark,
>
> Personally I don't think it's overly complicated, but it clearly is
> time consuming regardless of whether one uses your methodology or
> another. The question is, is it worth it ? I think the answer is
> self-evident.
>
> --- In amibroker@xxxxxxxxxxxxxxx, "quanttrader714"
> <quanttrader714@xxxx> wrote:
> > Not to be a naysayer, but... I agree with Howard in principle,
> > however, the devil's in the details. My view: the OOS I'm
concerned
> > with are actual trades. I personally consider existing data in
> sample
> > so applying what Howard suggested in this context is reconciling
> real
> > time results with expected results to see if the system is still
> > performing acceptably. And that's more difficult in practice than
> > developing robustness criteria and finding robust systems. IMO
the
> > results from criteria 3-5 are sufficient to use as expected
results
> if
> > done keeping the cautions I posted in 4 & 5 in mind and applying
an
> > appropriate adjustment to the simulations as I also mentioned in
> that
> > post because, as Howard said, OOS results are almost always less
> > profitable. Then there are several Statistical Process Control
> (SPC)
> > techniques that, given criteria 3-5 results, will give you insight
> > into whether or not the system's out of control. Or, given *a
lot*
> of
> > experience, one could even eyeball it. But it certainly ain't
> simple.
> >
> > Several have commented here and privately that all this seems
> awfully
> > complicated. So instead of another Edison quote, let me suggest
> > looking at (find w/google) some of Caxton's research (Bruce
Kovner,
> > first Market Wizards book, started by borrowing $3,000 and now
#111
> on
> > Forbes 400 list of wealthiest Americans).
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "Gary A. Serkhoshian"
> > <serkhoshian777@xxxx> wrote:
> > > Thanks Howard. Makes sense, and seems simple to implement.
With
> > Tomasz adding MCS into AmiBroker, life will only get sweeter : )
> > >
> > > Kind Regards,
> > > Gary
> > >
> > > Howard Bandy <howardbandy@xxxx> wrote:
> > >
> > > Hi Gary –
> > >
> > >
> > >
> > > I was thinking of looking at the recent trades in the out of
> sample
> > period. We can get an idea of what the possible distribution of
> > various metrics are by looking at the in sample results. But the
> out
> > of sample results are (almost) always less profitable, have a
lower
> > ratio of wins to losses, etc than the in sample results. One
> > technique I use is to run a quick and dirty monte carlo program I
> > wrote in Basic that gives the likelihood of various metrics –
> > such as
> > the proportion of winning versus losing trades. If the out of
> sample
> > results start falling in the area of the distribution that is
> > "unlikely", then I have a warning that the system may be
> > broken.
> > >
> > >
> > >
> > > Howard
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Gary A. Serkhoshian [mailto:serkhoshian777@x...]
> > > Sent: Sunday, November 09, 2003 11:33 AM
> > > To: amibroker@xxxxxxxxxxxxxxx
> > > Subject: RE: [amibroker] On Robustness, Post #1 : TO HOWARD
> > >
> > >
> > >
> > > Howard,
> > >
> > >
> > >
> > > Thanks for the detailed response. Helpful as always.
> > >
> > >
> > >
> > > Regarding your comment:
> > >
> > >
> > >
> > > Other techniques could be comparison of various metrics of
recent
> > trades with the probabilities that those results come from a
system
> > that is healthy or broken.
> > >
> > >
> > >
> > > Can we come to this conclusion by looking at frequency
> > distributions of the metrics in question during the IS period?
> > >
> > >
> > >
> > > Thanks again,
> > >
> > > Gary
> > >
> > >
> > >
> > >
> > >
> > > Howard Bandy <howardbandy@xxxx> wrote:
> > >
> > > Hi Gary –
> > >
> > >
> > >
> > > One -- Yes, the presidential election cycle has strong biases.
> > This year, year 3, is traditionally an up year. I mentioned the
> > presidential cycle as an example to ward off the flames that
might
> > come from suggesting a crystal ball approach to model selection.
> > >
> > >
> > >
> > > Two -- I have done quite a bit of research into use of
analysis
> of
> > the equity curve as a feedback mechanism to help determine the
> health
> > of a system. Other techniques could be comparison of various
> metrics
> > of recent trades with the probabilities that those results come
> from a
> > system that is healthy or broken.
> > >
> > >
> > >
> > > Three -- As I have mentioned in posts to this board and to
> > HolyGrailSM, I believe several things are true of markets and
> systems.
> > Not everyone on this list agrees with me on these points, so
> > you'll
> > read some other opinions..
> > >
> > > Entries and exits need not be symmetric. In the equity
> markets,
> > drops are much steeper in slope than rises, so the parameters
used
> to
> > recognize them in the same number of bars are different.
> > > A good system need not trade all, or even a large portion, of
> > tradables well.
> > > Markets change dramatically over time. It is very difficult
to
> > design a system that trades profitably over a long time period,
> > particularly when the market characteristics change within that
> > period.
> > > Systems that once worked well, then fail, will probably not
> work
> > well again.
> > >
> > >
> > >
> > > Howard
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Gary A. Serkhoshian [mailto:serkhoshian777@x...]
> > > Sent: Wednesday, November 05, 2003 9:15 PM
> > > To: amibroker@xxxxxxxxxxxxxxx
> > > Subject: RE: [amibroker] On Robustness, Post #1 : TO HOWARD
> > >
> > >
> > >
> > > Howard,
> > >
> > >
> > >
> > > A few questions regarding your post to Dave
> > >
> > >
> > >
> > > < One – we design two systems, one for bullish periods, the
> > other
> > for bearish periods. Then we look into our crystal ball and use
the
> > system with the upward bias if we have some idea that the near
> future
> > will be bullish, and use the system with the downward bias if we
> have
> > some idea that the near future will be bearish. This is not
wholly
> a
> > dream. For example, there are strong seasonalities in the US four
> > year presidential cycle. >
> > >
> > >
> > >
> > > Regarding the 4-year cycle, are you specifically referring to
> year 3
> > (the year that is coming to an end) ? I've read year 3 since
WWII
> has
> > been profitable to the tune of 15% on avg. Any other cycles you'd
> > suggest to follow?
> > >
> > >
> > >
> > > < Two – we design two systems, one for bullish periods, the
> > other
> > for bearish periods, and include failsafe mechanisms in both.
Then
> we
> > trade both systems and let the system that works make money while
> the
> > system that doesn't work recognizes that it doesn't work and
> > stays flat. >
> > >
> > >
> > >
> > > Which failsafe mechanisms do you prefer? I've been looking at
a
> DD
> > "floor" or perhaps a factor of MaxDD to turn off the system. I
> > believe Dimitris has suggested a downslope in the 100MA of equity
> > curve which makes sense, too.
> > >
> > >
> > >
> > > < Three – we design a system that is profitable in both
bull and
> > bear periods. I think this is the hardest to do, since the
markets
> > act so differently that it requires additional parameters to be
able
> > to recognize the additional patterns. In my experience, systems
> > designed to do well in both bullish and bearish periods do not do
> > exceptionally well in either period >
> > >
> > >
> > >
> > > What you're describing is essentially that buys and shorts can
not
> > be symmetrical. Is that right? What are the primary things that
> > differentiate up moves from down moves that require the need for
> > asymmetry of signals.
> > >
> > >
> > >
> > > Thanks for the post, as this subject is scratching where I itch
in
> > my eduction of system development and optmization.
> > >
> > >
> > >
> > > Kind Regards,
> > >
> > > Gary
> > >
> > >
> > >
> > > Howard Bandy <howardbandy@xxxx> wrote:
> > >
> > > Hi Dave –
> > >
> > >
> > >
> > > Good posting. I'd like to comment on your last paragraph.
> > >
> > >
> > >
> > > - if one system does better in bull years and another in bear,
the
> > one that
> > > does better in reality will depend on the proportion of bull and
> > bear years
> > > that actually occur. when we weight bull, bear and sideways
> markets
> > equally,
> > > are we matching their proportions in real life? what time frame
> > would we
> > > want to base that judgment on?
> > >
> > > It seems there are three approaches to take.
> > >
> > >
> > >
> > > One – we design two systems, one for bullish periods, the
other
> > for
> > bearish periods. Then we look into our crystal ball and use the
> > system with the upward bias if we have some idea that the near
> future
> > will be bullish, and use the system with the downward bias if we
> have
> > some idea that the near future will be bearish. This is not
wholly
> a
> > dream. For example, there are strong seasonalities in the US four
> > year presidential cycle.
> > >
> > >
> > >
> > > Two – we design two systems, one for bullish periods, the
other
> > for
> > bearish periods, and include failsafe mechanisms in both. Then we
> > trade both systems and let the system that works make money while
> the
> > system that doesn't work recognizes that it doesn't work and
> > stays flat.
> > >
> > >
> > >
> > > Three – we design a system that is profitable in both bull
and
> > bear
> > periods. I think this is the hardest to do, since the markets
act
> so
> > differently that it requires additional parameters to be able to
> > recognize the additional patterns. In my experience, systems
> designed
> > to do well in both bullish and bearish periods do not do
> exceptionally
> > well in either period.
> > >
> > >
> > >
> > > Howard
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Dave Merrill [mailto:dmerrill@x...]
> > > Sent: Monday, November 03, 2003 7:06 AM
> > > To: amibroker@xxxxxxxxxxxxxxx
> > > Subject: RE: [amibroker] On Robustness, Post #1
> > >
> > >
> > >
> > > some robustness issues that have been rattling around in my head
> > over the
> > > weekend...
> > >
> > > <<<SNIP>>>
> > >
> > >
> > >
> > >
> > >
> > > Send BUG REPORTS to bugs@xxxx
> > > Send SUGGESTIONS to suggest@xxxx
> > > -----------------------------------------
> > > Post AmiQuote-related messages ONLY to: amiq
uote@xxxxxxxxxxxxxxx
> > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > --------------------------------------------
> > > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > >
> > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> > Service.
> > >
> > >
> > >
> > >
> > >
> > >
> > > Howard Bandy <howardbandy@xxxx> wrote:
> > >
> > > Hi Dave –
> > >
> > >
> > >
> > > Good posting. I'd like to comment on your last paragraph.
> > >
> > >
> > >
> > > - if one system does better in bull years and another in bear,
the
> > one that
> > > does better in reality will depend on the proportion of bull and
> > bear years
> > > that actually occur. when we weight bull, bear and sideways
> markets
> > equally,
> > > are we matching their proportions in real life? what time frame
> > would we
> > > want to base that judgment on?
> > >
> > > It seems there are three approaches to take.
> > >
> > >
> > >
> > > One – we design two systems, one for bullish periods, the
other
> > for
> > bearish periods. Then we look into our crystal ball and use the
> > system with the upward bias if we have some idea that the near
> future
> > will be bullish, and use the system with the downward bias if we
> have
> > some idea that the near future will be bearish. This is not
wholly
> a
> > dream. For example, there are strong seasonalities in the US four
> > year presidential cycle.
> > >
> > >
> > >
> > > Two – we design two systems, one for bullish periods, the
other
> > for
> > bearish periods, and include failsafe mechanisms in both. Then we
> > trade both systems and let the system that works make money while
> the
> > system that doesn't work recognizes that it doesn't work and
> > stays flat.
> > >
> > >
> > >
> > > Three – we design a system that is profitable in both bull
and
> > bear
> > periods. I think this is the hardest to do, since the markets
act
> so
> > differently that it requires additional parameters to be able to
> > recognize the additional patterns. In my experience, systems
> designed
> > to do well in both bullish and bearish periods do not do
> exceptionally
> > well in either period.
> > >
> > >
> > >
> > > Howard
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Dave Merrill [mailto:dmerrill@x...]
> > > Sent: Monday, November 03, 2003 7:06 AM
> > > To: amibroker@xxxxxxxxxxxxxxx
> > > Subject: RE: [amibroker] On Robustness, Post #1
> > >
> > >
> > >
> > > some robustness issues that have been rattling around in my head
> > over the
> > > weekend...
> > >
> > > <<<SNIP>>>
> > >
> > >
> > >
> > >
> > >
> > >
> > > Send BUG REPORTS to bugs@xxxx
> > > Send SUGGESTIONS to suggest@xxxx
> > > -----------------------------------------
> > > Post AmiQuote-related messages ONLY to: amiq
uote@xxxxxxxxxxxxxxx
> > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > --------------------------------------------
> > > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > >
> > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> > Service.
> > >
> > > ---------------------------------
> > >
> > >
> > > Do you Yahoo!?
> > > Protect your identity with Yahoo! Mail AddressGuard
> > >
> > > Send BUG REPORTS to bugs@xxxx
> > > Send SUGGESTIONS to suggest@xxxx
> > > -----------------------------------------
> > > Post AmiQuote-related messages ONLY to: amiq
uote@xxxxxxxxxxxxxxx
> > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > --------------------------------------------
> > > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > >
> > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> > Service.
> > >
> > >
> > >
> > >
> > >
> > > Send BUG REPORTS to bugs@xxxx
> > > Send SUGGESTIONS to suggest@xxxx
> > > -----------------------------------------
> > > Post AmiQuote-related messages ONLY to: amiq
uote@xxxxxxxxxxxxxxx
> > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > --------------------------------------------
> > > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > >
> > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> > Service.
> > >
> > > ---------------------------------
> > >
> > >
> > > Do you Yahoo!?
> > > Protect your identity with Yahoo! Mail AddressGuard
> > >
> > >
> > > Send BUG REPORTS to bugs@xxxx
> > > Send SUGGESTIONS to suggest@xxxx
> > > -----------------------------------------
> > > Post AmiQuote-related messages ONLY to: amiq
uote@xxxxxxxxxxxxxxx
> > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > --------------------------------------------
> > > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > >
> > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> > Service.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > Yahoo! Groups SponsorADVERTISEMENT
> > >
> > > Send BUG REPORTS to bugs@xxxx
> > > Send SUGGESTIONS to suggest@xxxx
> > > -----------------------------------------
> > > Post AmiQuote-related messages ONLY to: amiq
uote@xxxxxxxxxxxxxxx
> > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > --------------------------------------------
> > > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > >
> > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> > Service.
> > >
> > >
> > > ---------------------------------
> > > Do you Yahoo!?
> > > Protect your identity with Yahoo! Mail AddressGuard
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|