|
PureBytes Links
Trading Reference Links
|
Mark,
Thanks for pointing me in the right direction re Kovner and Caxton. It wasn't easy to find info on Caxton, but did find quite a few articles by Bary Schachter (Caxton Corp) on Stress Testing. His comment below really struck me, and is essentially what you and Fred have alluded to.
http://www.erisk.com/LearningCenter/Jigsaw/market_schacter.asp
Stress TestingBarry Schachter of Caxton Corporation and Gloria Mundi
Technology is making it easy to construct bad stress tests. Off-the-shelf risk management software comes with (relatively) easy-to-use graphical user interfaces for tweaking rates and prices. While it's a boon to have the ability to respond quickly when constructing stress scenarios in response to rapidly changing conditions, velocity is no substitute for perspicacity. Stress testing demands a serious commitment of the right kind of resources, and because of this need we may see that firms begin to outsource stress test construction.
For those not familiar, I found www.gloriamundi.org to be a good resource. www.erisk.com also carries a lot of Schachter’s writings. I’ve attached one article I found helpful by Schachter entitled “Stress Testing for Fun and Profit”.
Mark, thanks for the patience, and I guess I’ll go find my overalls and try again to get my arms around your criteria. The more I learn, the more I realize there is still a lot I don’t know.
Kind Regards,
Gary
quanttrader714 <quanttrader714@xxxxxxxxx> wrote:
For the robustness stuff I posted and many other methodologies, I'dagree. But with all due respect to Gary, what made me want torespond was that if one didn't already know the info in Howard's postand if one has little if any experience with MCS there are lots ofpitfalls, especially with trying to bridge the gap between expectedand actual realtime results. And trying to do simulations withoutbeing aware of them and ways to mitigate gives poor results and a badname to a perfectly valid and useful process. --- In amibroker@xxxxxxxxxxxxxxx, "Fred" <fctonetti@xxxx> wrote:> Mark,> > Personally I don't think it's overly complicated, but it clearly is > time consuming regardless of whether one uses your methodology or > another. The question is, is it worth it ? I think the answer is
> self-evident.> > --- In amibroker@xxxxxxxxxxxxxxx, "quanttrader714" > <quanttrader714@xxxx> wrote:> > Not to be a naysayer, but... I agree with Howard in principle,> > however, the devil's in the details. My view: the OOS I'mconcerned> > with are actual trades. I personally consider existing data in > sample> > so applying what Howard suggested in this context is reconciling > real> > time results with expected results to see if the system is still> > performing acceptably. And that's more difficult in practice than> > developing robustness criteria and finding robust systems. IMOthe> > results from criteria 3-5 are sufficient to use as expectedresults > if> > done keeping the cautions I posted in 4 & 5 in mind and applyingan> > appropriate adjustment to the simulations as I also
mentioned in > that> > post because, as Howard said, OOS results are almost always less> > profitable. Then there are several Statistical Process Control > (SPC)> > techniques that, given criteria 3-5 results, will give you insight> > into whether or not the system's out of control. Or, given *alot* > of> > experience, one could even eyeball it. But it certainly ain't > simple.> > > > Several have commented here and privately that all this seems > awfully> > complicated. So instead of another Edison quote, let me suggest> > looking at (find w/google) some of Caxton's research (BruceKovner,> > first Market Wizards book, started by borrowing $3,000 and now#111 > on> > Forbes 400 list of wealthiest Americans). > > > > --- In amibroker@xxxxxxxxxxxxxxx, "Gary A.
Serkhoshian"> > <serkhoshian777@xxxx> wrote:> > > Thanks Howard. Makes sense, and seems simple to implement. With> > Tomasz adding MCS into AmiBroker, life will only get sweeter : )> > > > > > Kind Regards,> > > Gary> > > > > > Howard Bandy <howardbandy@xxxx> wrote:> > > > > > Hi Gary –> > > > > > > > > > > > I was thinking of looking at the recent trades in the out of > sample> > period. We can get an idea of what the possible distribution of> > various metrics are by looking at the in sample results. But the > out> > of sample results are (almost) always less profitable, have alower> > ratio of wins to losses, etc than the in sample results. One> > technique I use is to run a quick and
dirty monte carlo program I> > wrote in Basic that gives the likelihood of various metrics –> > such as> > the proportion of winning versus losing trades. If the out of > sample> > results start falling in the area of the distribution that is> > "unlikely", then I have a warning that the system may be> > broken.> > > > > > > > > > > > Howard> > > > > > > > > > > > -----Original Message-----> > > From: Gary A. Serkhoshian [mailto:serkhoshian777@xxxx] > > > Sent: Sunday, November 09, 2003 11:33 AM> > > To: amibroker@xxxxxxxxxxxxxxx> > > Subject: RE: [amibroker] On Robustness, Post #1 : TO HOWARD> > > > > > > > > > > > Howard,> > > > > > > > >
> > > Thanks for the detailed response. Helpful as always.> > > > > > > > > > > > Regarding your comment:> > > > > > > > > > > > Other techniques could be comparison of various metrics ofrecent> > trades with the probabilities that those results come from asystem> > that is healthy or broken.> > > > > > > > > > > > Can we come to this conclusion by looking at frequency> > distributions of the metrics in question during the IS period?> > > > > > > > > > > > Thanks again,> > > > > > Gary> > > > > > > > > > > > > > > > > > Howard Bandy <howardbandy@xxxx> wrote:> > >
> > > Hi Gary –> > > > > > > > > > > > One -- Yes, the presidential election cycle has strong biases. > > This year, year 3, is traditionally an up year. I mentioned the> > presidential cycle as an example to ward off the flames thatmight> > come from suggesting a crystal ball approach to model selection. > > > > > > > > > > > > Two -- I have done quite a bit of research into use ofanalysis > of> > the equity curve as a feedback mechanism to help determine the > health> > of a system. Other techniques could be comparison of various > metrics> > of recent trades with the probabilities that those results come > from a> > system that is healthy or broken.> > > > > > > > >
> > > Three -- As I have mentioned in posts to this board and to> > HolyGrailSM, I believe several things are true of markets and > systems.> > Not everyone on this list agrees with me on these points, so> > you'll> > read some other opinions.. > > > > > > Entries and exits need not be symmetric. In the equity > markets,> > drops are much steeper in slope than rises, so the parametersused > to> > recognize them in the same number of bars are different. > > > A good system need not trade all, or even a large portion, of> > tradables well.> > > Markets change dramatically over time. It is very difficultto> > design a system that trades profitably over a long time period,> > particularly when the market
characteristics change within that> > period.> > > Systems that once worked well, then fail, will probably not > work> > well again.> > > > > > > > > > > > Howard> > > > > > > > > > > > -----Original Message-----> > > From: Gary A. Serkhoshian [mailto:serkhoshian777@xxxx] > > > Sent: Wednesday, November 05, 2003 9:15 PM> > > To: amibroker@xxxxxxxxxxxxxxx> > > Subject: RE: [amibroker] On Robustness, Post #1 : TO HOWARD> > > > > > > > > > > > Howard,> > > > > > > > > > > > A few questions regarding your post to Dave> > > > > > > > > > > > < One – we design two systems, one for bullish
periods, the> > other> > for bearish periods. Then we look into our crystal ball and usethe> > system with the upward bias if we have some idea that the near > future> > will be bullish, and use the system with the downward bias if we > have> > some idea that the near future will be bearish. This is notwholly > a> > dream. For example, there are strong seasonalities in the US four> > year presidential cycle. >> > > > > > > > > > > > Regarding the 4-year cycle, are you specifically referring to > year 3> > (the year that is coming to an end) ? I've read year 3 sinceWWII > has> > been profitable to the tune of 15% on avg. Any other cycles you'd> > suggest to follow?> > > > > > > > > > > > <
Two – we design two systems, one for bullish periods, the> > other> > for bearish periods, and include failsafe mechanisms in both. Then > we> > trade both systems and let the system that works make money while > the> > system that doesn't work recognizes that it doesn't work and> > stays flat. >> > > > > > > > > > > > Which failsafe mechanisms do you prefer? I've been looking ata > DD> > "floor" or perhaps a factor of MaxDD to turn off the system. I> > believe Dimitris has suggested a downslope in the 100MA of equity> > curve which makes sense, too.> > > > > > > > > > > > < Three – we design a system that is profitable in bothbull and> > bear periods. I think this is the hardest to do, since themarkets> > act
so differently that it requires additional parameters to beable> > to recognize the additional patterns. In my experience, systems> > designed to do well in both bullish and bearish periods do not do> > exceptionally well in either period >> > > > > > > > > > > > What you're describing is essentially that buys and shorts cannot> > be symmetrical. Is that right? What are the primary things that> > differentiate up moves from down moves that require the need for> > asymmetry of signals.> > > > > > > > > > > > Thanks for the post, as this subject is scratching where I itchin> > my eduction of system development and optmization.> > > > > > > > > > > > Kind Regards,> > > > > > Gary>
> > > > > > > > > > > Howard Bandy <howardbandy@xxxx> wrote:> > > > > > Hi Dave –> > > > > > > > > > > > Good posting. I'd like to comment on your last paragraph.> > > > > > > > > > > > - if one system does better in bull years and another in bear,the> > one that> > > does better in reality will depend on the proportion of bull and> > bear years> > > that actually occur. when we weight bull, bear and sideways > markets> > equally,> > > are we matching their proportions in real life? what time frame> > would we> > > want to base that judgment on?> > > > > > It seems there are three approaches to take. > > > > > > >
> > > > > One – we design two systems, one for bullish periods, theother> > for> > bearish periods. Then we look into our crystal ball and use the> > system with the upward bias if we have some idea that the near > future> > will be bullish, and use the system with the downward bias if we > have> > some idea that the near future will be bearish. This is notwholly > a> > dream. For example, there are strong seasonalities in the US four> > year presidential cycle.> > > > > > > > > > > > Two – we design two systems, one for bullish periods, theother> > for> > bearish periods, and include failsafe mechanisms in both. Then we> > trade both systems and let the system that works make money while > the> > system that doesn't work recognizes
that it doesn't work and> > stays flat.> > > > > > > > > > > > Three – we design a system that is profitable in both bulland> > bear> > periods. I think this is the hardest to do, since the marketsact > so> > differently that it requires additional parameters to be able to> > recognize the additional patterns. In my experience, systems > designed> > to do well in both bullish and bearish periods do not do > exceptionally> > well in either period.> > > > > > > > > > > > Howard> > > > > > > > > > > > -----Original Message-----> > > From: Dave Merrill [mailto:dmerrill@xxxx] > > > Sent: Monday, November 03, 2003 7:06 AM> > > To: amibroker@xxxxxxxxxxxxxxx> >
> Subject: RE: [amibroker] On Robustness, Post #1> > > > > > > > > > > > some robustness issues that have been rattling around in my head> > over the> > > weekend...> > > > > > <<<SNIP>>> > > > > > > > > > > > > > > > > > > Send BUG REPORTS to bugs@xxxx> > > Send SUGGESTIONS to suggest@xxxx> > > -----------------------------------------> > > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx > > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)> > > --------------------------------------------> > > Check group FAQ at:> > <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html > > > > > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of> > Service. > > > > > > > > > > > > > > > > > > > > > Howard Bandy <howardbandy@xxxx> wrote:> > > > > > Hi Dave –> > > > > > > > > > > > Good posting. I'd like to comment on your last paragraph.> > > > > > > > > > > > - if one system does better in bull years and another in bear,the> > one that> > > does better in reality will depend on the proportion of bull and> > bear years> > > that actually occur. when we weight bull, bear and sideways >
markets> > equally,> > > are we matching their proportions in real life? what time frame> > would we> > > want to base that judgment on?> > > > > > It seems there are three approaches to take. > > > > > > > > > > > > One – we design two systems, one for bullish periods, theother> > for> > bearish periods. Then we look into our crystal ball and use the> > system with the upward bias if we have some idea that the near > future> > will be bullish, and use the system with the downward bias if we > have> > some idea that the near future will be bearish. This is notwholly > a> > dream. For example, there are strong seasonalities in the US four> > year presidential cycle.> > > > > > > > > >
> > Two – we design two systems, one for bullish periods, theother> > for> > bearish periods, and include failsafe mechanisms in both. Then we> > trade both systems and let the system that works make money while > the> > system that doesn't work recognizes that it doesn't work and> > stays flat.> > > > > > > > > > > > Three – we design a system that is profitable in both bulland> > bear> > periods. I think this is the hardest to do, since the marketsact > so> > differently that it requires additional parameters to be able to> > recognize the additional patterns. In my experience, systems > designed> > to do well in both bullish and bearish periods do not do > exceptionally> > well in either period.> > > > > > > >
> > > > Howard> > > > > > > > > > > > -----Original Message-----> > > From: Dave Merrill [mailto:dmerrill@xxxx] > > > Sent: Monday, November 03, 2003 7:06 AM> > > To: amibroker@xxxxxxxxxxxxxxx> > > Subject: RE: [amibroker] On Robustness, Post #1> > > > > > > > > > > > some robustness issues that have been rattling around in my head> > over the> > > weekend...> > > > > > <<<SNIP>>> > > > > > > > > > > > > > > > > > > > > > Send BUG REPORTS to bugs@xxxx> > > Send SUGGESTIONS to suggest@xxxx> > > -----------------------------------------> > > Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx > > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)> > > --------------------------------------------> > > Check group FAQ at:> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html > > > > > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of> > Service. > > > > > > ---------------------------------> > > > > > > > > Do you Yahoo!?> > > Protect your identity with Yahoo! Mail AddressGuard > > > > > > Send BUG REPORTS to bugs@xxxx> > > Send SUGGESTIONS to suggest@xxxx> > > -----------------------------------------> > > Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx > > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)> > > --------------------------------------------> > > Check group FAQ at:> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html > > > > > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of> > Service. > > > > > > > > > > > > > > > > > > Send BUG REPORTS to bugs@xxxx> > > Send SUGGESTIONS to suggest@xxxx> > > -----------------------------------------> > > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx > > > (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)> > > --------------------------------------------> > > Check group FAQ at:> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html > > > > > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of> > Service. > > > > > > ---------------------------------> > > > > > > > > Do you Yahoo!?> > > Protect your identity with Yahoo! Mail AddressGuard > > > > > > > > > Send BUG REPORTS to bugs@xxxx> > > Send SUGGESTIONS to suggest@xxxx> > > -----------------------------------------> > > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx > > >
(Web page: http://groups.yahoo.com/group/amiquote/messages/)> > > --------------------------------------------> > > Check group FAQ at:> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html > > > > > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of> > Service. > > > > > > > > > > > > > > > > > > > > > > > > Yahoo! Groups SponsorADVERTISEMENT> > > > > > Send BUG REPORTS to bugs@xxxx> > > Send SUGGESTIONS to suggest@xxxx> > > -----------------------------------------> > > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx > > > (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)> > > --------------------------------------------> > > Check group FAQ at:> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html > > > > > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of> > Service. > > > > > > > > > ---------------------------------> > > Do you Yahoo!?> > > Protect your identity with Yahoo! Mail AddressGuardSend BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to suggest@xxxxxxxxxxxxx-----------------------------------------Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
Yahoo! Groups Sponsor
ADVERTISEMENT
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
Attachment:
Stressed.pdf
Attachment:
Description: "Description: Stressed.pdf"
|