|
PureBytes Links
Trading Reference Links
|
Mark,
Personally I don't think it's overly complicated, but it clearly is
time consuming regardless of whether one uses your methodology or
another. The question is, is it worth it ? I think the answer is
self-evident.
--- In amibroker@xxxxxxxxxxxxxxx, "quanttrader714"
<quanttrader714@xxxx> wrote:
> Not to be a naysayer, but... I agree with Howard in principle,
> however, the devil's in the details. My view: the OOS I'm concerned
> with are actual trades. I personally consider existing data in
sample
> so applying what Howard suggested in this context is reconciling
real
> time results with expected results to see if the system is still
> performing acceptably. And that's more difficult in practice than
> developing robustness criteria and finding robust systems. IMO the
> results from criteria 3-5 are sufficient to use as expected results
if
> done keeping the cautions I posted in 4 & 5 in mind and applying an
> appropriate adjustment to the simulations as I also mentioned in
that
> post because, as Howard said, OOS results are almost always less
> profitable. Then there are several Statistical Process Control
(SPC)
> techniques that, given criteria 3-5 results, will give you insight
> into whether or not the system's out of control. Or, given *a lot*
of
> experience, one could even eyeball it. But it certainly ain't
simple.
>
> Several have commented here and privately that all this seems
awfully
> complicated. So instead of another Edison quote, let me suggest
> looking at (find w/google) some of Caxton's research (Bruce Kovner,
> first Market Wizards book, started by borrowing $3,000 and now #111
on
> Forbes 400 list of wealthiest Americans).
>
> --- In amibroker@xxxxxxxxxxxxxxx, "Gary A. Serkhoshian"
> <serkhoshian777@xxxx> wrote:
> > Thanks Howard. Makes sense, and seems simple to implement. With
> Tomasz adding MCS into AmiBroker, life will only get sweeter : )
> >
> > Kind Regards,
> > Gary
> >
> > Howard Bandy <howardbandy@xxxx> wrote:
> >
> > Hi Gary –
> >
> >
> >
> > I was thinking of looking at the recent trades in the out of
sample
> period. We can get an idea of what the possible distribution of
> various metrics are by looking at the in sample results. But the
out
> of sample results are (almost) always less profitable, have a lower
> ratio of wins to losses, etc than the in sample results. One
> technique I use is to run a quick and dirty monte carlo program I
> wrote in Basic that gives the likelihood of various metrics –
> such as
> the proportion of winning versus losing trades. If the out of
sample
> results start falling in the area of the distribution that is
> "unlikely", then I have a warning that the system may be
> broken.
> >
> >
> >
> > Howard
> >
> >
> >
> > -----Original Message-----
> > From: Gary A. Serkhoshian [mailto:serkhoshian777@x...]
> > Sent: Sunday, November 09, 2003 11:33 AM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: RE: [amibroker] On Robustness, Post #1 : TO HOWARD
> >
> >
> >
> > Howard,
> >
> >
> >
> > Thanks for the detailed response. Helpful as always.
> >
> >
> >
> > Regarding your comment:
> >
> >
> >
> > Other techniques could be comparison of various metrics of recent
> trades with the probabilities that those results come from a system
> that is healthy or broken.
> >
> >
> >
> > Can we come to this conclusion by looking at frequency
> distributions of the metrics in question during the IS period?
> >
> >
> >
> > Thanks again,
> >
> > Gary
> >
> >
> >
> >
> >
> > Howard Bandy <howardbandy@xxxx> wrote:
> >
> > Hi Gary –
> >
> >
> >
> > One -- Yes, the presidential election cycle has strong biases.
> This year, year 3, is traditionally an up year. I mentioned the
> presidential cycle as an example to ward off the flames that might
> come from suggesting a crystal ball approach to model selection.
> >
> >
> >
> > Two -- I have done quite a bit of research into use of analysis
of
> the equity curve as a feedback mechanism to help determine the
health
> of a system. Other techniques could be comparison of various
metrics
> of recent trades with the probabilities that those results come
from a
> system that is healthy or broken.
> >
> >
> >
> > Three -- As I have mentioned in posts to this board and to
> HolyGrailSM, I believe several things are true of markets and
systems.
> Not everyone on this list agrees with me on these points, so
> you'll
> read some other opinions..
> >
> > Entries and exits need not be symmetric. In the equity
markets,
> drops are much steeper in slope than rises, so the parameters used
to
> recognize them in the same number of bars are different.
> > A good system need not trade all, or even a large portion, of
> tradables well.
> > Markets change dramatically over time. It is very difficult to
> design a system that trades profitably over a long time period,
> particularly when the market characteristics change within that
> period.
> > Systems that once worked well, then fail, will probably not
work
> well again.
> >
> >
> >
> > Howard
> >
> >
> >
> > -----Original Message-----
> > From: Gary A. Serkhoshian [mailto:serkhoshian777@x...]
> > Sent: Wednesday, November 05, 2003 9:15 PM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: RE: [amibroker] On Robustness, Post #1 : TO HOWARD
> >
> >
> >
> > Howard,
> >
> >
> >
> > A few questions regarding your post to Dave
> >
> >
> >
> > < One – we design two systems, one for bullish periods, the
> other
> for bearish periods. Then we look into our crystal ball and use the
> system with the upward bias if we have some idea that the near
future
> will be bullish, and use the system with the downward bias if we
have
> some idea that the near future will be bearish. This is not wholly
a
> dream. For example, there are strong seasonalities in the US four
> year presidential cycle. >
> >
> >
> >
> > Regarding the 4-year cycle, are you specifically referring to
year 3
> (the year that is coming to an end) ? I've read year 3 since WWII
has
> been profitable to the tune of 15% on avg. Any other cycles you'd
> suggest to follow?
> >
> >
> >
> > < Two – we design two systems, one for bullish periods, the
> other
> for bearish periods, and include failsafe mechanisms in both. Then
we
> trade both systems and let the system that works make money while
the
> system that doesn't work recognizes that it doesn't work and
> stays flat. >
> >
> >
> >
> > Which failsafe mechanisms do you prefer? I've been looking at a
DD
> "floor" or perhaps a factor of MaxDD to turn off the system. I
> believe Dimitris has suggested a downslope in the 100MA of equity
> curve which makes sense, too.
> >
> >
> >
> > < Three – we design a system that is profitable in both bull and
> bear periods. I think this is the hardest to do, since the markets
> act so differently that it requires additional parameters to be able
> to recognize the additional patterns. In my experience, systems
> designed to do well in both bullish and bearish periods do not do
> exceptionally well in either period >
> >
> >
> >
> > What you're describing is essentially that buys and shorts can not
> be symmetrical. Is that right? What are the primary things that
> differentiate up moves from down moves that require the need for
> asymmetry of signals.
> >
> >
> >
> > Thanks for the post, as this subject is scratching where I itch in
> my eduction of system development and optmization.
> >
> >
> >
> > Kind Regards,
> >
> > Gary
> >
> >
> >
> > Howard Bandy <howardbandy@xxxx> wrote:
> >
> > Hi Dave –
> >
> >
> >
> > Good posting. I'd like to comment on your last paragraph.
> >
> >
> >
> > - if one system does better in bull years and another in bear, the
> one that
> > does better in reality will depend on the proportion of bull and
> bear years
> > that actually occur. when we weight bull, bear and sideways
markets
> equally,
> > are we matching their proportions in real life? what time frame
> would we
> > want to base that judgment on?
> >
> > It seems there are three approaches to take.
> >
> >
> >
> > One – we design two systems, one for bullish periods, the other
> for
> bearish periods. Then we look into our crystal ball and use the
> system with the upward bias if we have some idea that the near
future
> will be bullish, and use the system with the downward bias if we
have
> some idea that the near future will be bearish. This is not wholly
a
> dream. For example, there are strong seasonalities in the US four
> year presidential cycle.
> >
> >
> >
> > Two – we design two systems, one for bullish periods, the other
> for
> bearish periods, and include failsafe mechanisms in both. Then we
> trade both systems and let the system that works make money while
the
> system that doesn't work recognizes that it doesn't work and
> stays flat.
> >
> >
> >
> > Three – we design a system that is profitable in both bull and
> bear
> periods. I think this is the hardest to do, since the markets act
so
> differently that it requires additional parameters to be able to
> recognize the additional patterns. In my experience, systems
designed
> to do well in both bullish and bearish periods do not do
exceptionally
> well in either period.
> >
> >
> >
> > Howard
> >
> >
> >
> > -----Original Message-----
> > From: Dave Merrill [mailto:dmerrill@x...]
> > Sent: Monday, November 03, 2003 7:06 AM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: RE: [amibroker] On Robustness, Post #1
> >
> >
> >
> > some robustness issues that have been rattling around in my head
> over the
> > weekend...
> >
> > <<<SNIP>>>
> >
> >
> >
> >
> >
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> Service.
> >
> >
> >
> >
> >
> >
> > Howard Bandy <howardbandy@xxxx> wrote:
> >
> > Hi Dave –
> >
> >
> >
> > Good posting. I'd like to comment on your last paragraph.
> >
> >
> >
> > - if one system does better in bull years and another in bear, the
> one that
> > does better in reality will depend on the proportion of bull and
> bear years
> > that actually occur. when we weight bull, bear and sideways
markets
> equally,
> > are we matching their proportions in real life? what time frame
> would we
> > want to base that judgment on?
> >
> > It seems there are three approaches to take.
> >
> >
> >
> > One – we design two systems, one for bullish periods, the other
> for
> bearish periods. Then we look into our crystal ball and use the
> system with the upward bias if we have some idea that the near
future
> will be bullish, and use the system with the downward bias if we
have
> some idea that the near future will be bearish. This is not wholly
a
> dream. For example, there are strong seasonalities in the US four
> year presidential cycle.
> >
> >
> >
> > Two – we design two systems, one for bullish periods, the other
> for
> bearish periods, and include failsafe mechanisms in both. Then we
> trade both systems and let the system that works make money while
the
> system that doesn't work recognizes that it doesn't work and
> stays flat.
> >
> >
> >
> > Three – we design a system that is profitable in both bull and
> bear
> periods. I think this is the hardest to do, since the markets act
so
> differently that it requires additional parameters to be able to
> recognize the additional patterns. In my experience, systems
designed
> to do well in both bullish and bearish periods do not do
exceptionally
> well in either period.
> >
> >
> >
> > Howard
> >
> >
> >
> > -----Original Message-----
> > From: Dave Merrill [mailto:dmerrill@x...]
> > Sent: Monday, November 03, 2003 7:06 AM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: RE: [amibroker] On Robustness, Post #1
> >
> >
> >
> > some robustness issues that have been rattling around in my head
> over the
> > weekend...
> >
> > <<<SNIP>>>
> >
> >
> >
> >
> >
> >
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> Service.
> >
> > ---------------------------------
> >
> >
> > Do you Yahoo!?
> > Protect your identity with Yahoo! Mail AddressGuard
> >
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> Service.
> >
> >
> >
> >
> >
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> Service.
> >
> > ---------------------------------
> >
> >
> > Do you Yahoo!?
> > Protect your identity with Yahoo! Mail AddressGuard
> >
> >
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> Service.
> >
> >
> >
> >
> >
> >
> >
> > Yahoo! Groups SponsorADVERTISEMENT
> >
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> Service.
> >
> >
> > ---------------------------------
> > Do you Yahoo!?
> > Protect your identity with Yahoo! Mail AddressGuard
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|