|
PureBytes Links
Trading Reference Links
|
Mark,
I read your last two posts twice, and am having a tough time
understanding what you're saying... You said following in your first
post -
> Then backtest on a random sample of issues drawn from the
> population of all stocks trading at the *start* of the period
> you're backtesting on. And oh, by the way, don't use *any* of
> this data to develop your system(s) because it theoretically
> didn't exist when you created them.
What do you mean by "don't use any of this data..."? And what do you
mean it didn't exist when you created them? What is "them"?
And in this last post -
> benchmark for *select* stocks from the population of stocks trading
at
> the start of the analysis period. I say select because, as you
know,
> N100 stocks are limited to non-financial companies listed on
NASDAQ.
If the point is to limit your trading to N100 stocks, what's the
point in making sure that the backtest *also* works equally well on
some other (non-N100) select stocks? No doubt, if you can make both
happen (i.e. make it work equally well on random stocks as well as
N100 stocks), then it's obviously great. But then you're effectively
eliminating any system that might work (for whatever reason) only
when it comes to N100 stocks, but not when entire universe is
considered. Are you saying such a beast can't exist? I, for one,
might lean towards agreeing with it, especially given Chuck's
comments as well as my own findings, but I hardly think one can
*conclusively* say so. Besides, if one really created accurate N100
lists for past data, and develops a system that seems to trade them
well in the past, should it be tossed out?
I guess I'm trying to understand if your basic argument is -- any
system that backtests only using a group of stocks such as N100, no
matter how accurate the data and lists were, are not something one
should trade, because they were not backtested on non-N100 stocks.
Wasn't entirely clear if that's what you're saying.
TIA,
Jitu
--- In amibroker@xxxxxxxxxxxxxxx, "quanttrader714"
<quanttrader714@xxxx> wrote:
> Chuck,
>
> I've been following this thread and first let me thank you for
> sharing. It's always good to see a reminder that trading isn't
easy,
> especially from someone with your experience. This is a complex
topic
> with many moving parts and I think that different people have
> different things in mind when discussing it (myself included, lol).
> You mentioned the current N100 list "success story" and I'd quantify
> this by looking at a benchmark for measuring it that is *unaffected
> by distortion from a system* (such as buy and hold profit) vs the
same
> benchmark for *select* stocks from the population of stocks trading
at
> the start of the analysis period. I say select because, as you
know,
> N100 stocks are limited to non-financial companies listed on
NASDAQ.
> Then there's the broader issue of survivorship bias which I'd also
> quantify without system distortion but can be avoided by acting only
> with information known at the time. What I *think* you did was try
to
> simulate the past with a one-frame-a-year rolling window of the
> N100 for your N100 systems (whatever they are). I'm not sure what
> your results show about the N100 list success story or survivorship
> bias because of all the confounding variables that what I think
you've
> done introduces. But what I personally take away is another
argument
> for trading robust systems.
>
> Mark
>
> --- In amibroker@xxxxxxxxxxxxxxx, "Chuck Rademacher"
> <chuck_rademacher@x> wrote:
> > Jayson,
> >
> > Where do I begin? First, let me put your mind at ease and tell
you
> that I
> > am successfully trading numerous systems against the entire
> universe of
> > stocks. Suffice to say that I have a 40-year track record
trading
> my own
> > as well as other people's money. As I mentioned in my earlier
> email, I
> > became frustrated by the fact that many people in this group were
> getting
> > such good backtested results from systems applied to the N100.
> >
> > My N100 systems being discussed performed very well in the last
> year, not
> > poorly as you suggest. They performed very well for several
years
> against
> > the CURRENT N100 list. But, the CURRENT N100 list is a success
> story for
> > those 100 stocks.
> >
> > It is my belief that selecting and ranking based on only 100
stocks
> simply
> > won't yield enough qualified buy signals to give good returns
> without large
> > drawdowns. By the time I add liquidity and price filtering,
there
> may only
> > be one or two signals at each swing of the overall market. That
> simply
> > doesn't give me an acceptable level of diversfication. Due to
the
> size of
> > the funds that I manage, I need to get off a fair amount of
dollars
> in a
> > fair number of positions in each system in order to play the game.
> >
> > I'll probably just go back to developing and trading systems that
> look at
> > the entire universe, rather than the N100. However, I will
suffer
> every
> > time I hear of someone seeing 2,500% annual returns in backtesting
> some new
> > system on the N100.
> > -----Original Message-----
> > From: Jayson [mailto:jcasavant@x...]
> > Sent: Wednesday, December 31, 2003 3:10 AM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: RE: [amibroker] The demise of 23 systems...
> >
> >
> > Chuck,
> > Thanks for the detailed reply. You certainly have more
experience
> system
> > testing then I but it just seems that if a system "Works" it
should
> work on
> > any given data set at any given time. By this I mean that there is
> really
> > nothing magical about the NAS100 or the SP500, they are just
stocks
> chosen,
> > for a variety of reasons, to be components of an index. If the
> system looks
> > at market timing components then a stock will either pass/fail
based
> on that
> > criteria. The same will be true for any additional components in
the
> system.
> > Why would stock "A" perform favorably while stock "B" does not,
> after all it
> > passed the same (series) of tests.
> >
> > Time is also a component of testing that tends to confuse me.
If a
> system
> > does poorly over the last 12 months why would you want to test it
> over a
> > longer period? True the system may produce favorable results if
you
> had run
> > it for 10 years but if it is failing for the last year do you
really
> want to
> > be following it? Wouldn't you instead want to be looking for a
> system that
> > is behaving better today? If your system begins to fail wouldn't
you
> want to
> > stop trading it until such time as it begins behaving again? You
may
> miss
> > out on some winning trades but have you not saved your self some
> losers in
> > the process? I have seen posts over the years that indicate some
are
> trying
> > to use MA's of the equity line as a filter to determine adherence
to
> a given
> > system. On the surface this seems to make sense to me. Have you
> done any
> > work in this area?
> >
> > In answer 4 you state that the system will not optimize on the
> revised
> > data. Does this not indicate that the system only worked on the
> stocks
> > originally tested and therefore its success/failure was actually
> random?
> > Pure luck based on the trades taken in the origininal data?
> >
> > Over the years I have tested various systems, some simple, some
> complex,
> > and have found the process frustrating. I can build a system that
> trades
> > well on a given universe of stocks. It will trade well on the the
> timeframe
> > I built the system on but when tested over a different time frame
or
> on a
> > different basket will invariably fall apart. It seems to me there
is
> > certainly an element of luck involved. Perhaps it is a coup[le of
> stocks in
> > the universe the have steadily produced, take them out and ...no
> joy.
> > Perhaps a large percentage in the group had a stellar couple of
> years. Test
> > out side that run up and again...no joy
> >
> > In your follow up you state that you have had little luck with
any
> system
> > testing less than 1,000 stocks. Do your systems take significantly
> more
> > trades when tested against a large group or are you simply trading
> stocks
> > meeting higher standards based on some type of ranking? If the
> latter than
> > perhaps stock selection is key. There certainly is a large group
of
> > believers in this theory over at IBD and at QP/HGS.
> >
> > I am sorry to hear your story as I am certain you have invested
> lots of
> > sweat equity into the development of your many systems. I
sincerely
> hope
> > that next year at this time you can share with us your success
> story.
> >
> >
> > Regards,
> > Jayson
> > -----Original Message-----
> > From: Chuck Rademacher [mailto:chuck_rademacher@x]
> > Sent: Tuesday, December 30, 2003 12:54 PM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: RE: [amibroker] The demise of 23 systems...
> >
> >
> > Jayson,
> >
> > I can tell you, but probably not convince you, that the systems
> were not
> > over optimised. However, let me tell you the following so you
can
> decide
> > for yourself:
> >
> > 1. All 23 of these systems were long only.
> >
> > 2. Each system had between 3 and 6 parameters, including two
for
> market
> > timing.
> >
> > 3. The performance of these systems did not fall off a cliff if
> any or
> > all of those parameters deviated in either direction.
> >
> > 4. Let's say that they were over optimised. The systems won't
> optimise
> > with the "revised" data.
> >
> >
> > I have to tell you a couple more things:
> >
> > 1. I tend to use exactly the same two-parameter market timing
> methodology
> > (including parameter values) for more than 120 systems.
> >
> > 2. I find that the more stocks I pump into a system, the more
it
> has to
> > select from and the better the performance.
> >
> > 3. I could never get any systems to work to my satisfaction on
> less than
> > 1,000 stocks.
> >
> > 4. I certainly could never get any system to work to my
> satisfaction on
> > the NASDAQ 100.
> >
> > 5. All of my systems work (40+ years) was previously done
> outside of
> > AmiBroker where my database knows which stocks belong to which
index
> at any
> > given minute in time.
> >
> > 6. I became frustrated with reports of numerous AmiBroker users
> having
> > success with NASDAQ 100 trading systems.
> >
> > 7. I created a (current) NASDAQ 100 watchlist and a bunch of
> systems that
> > appeared to work.
> >
> > 8. I'm actually trading 11 of the 23 systems that I
mentioned.
> Well, to
> > be honest, I was trading them until yesterday.
> >
> > 9. Once I tried the more accurate method of determining which
> stocks were
> > NASDAQ 100 constituents in AmiBroker, these systems fell apart.
> >
> > That's my story... back to the drawing board (unless I want to
> ignore
> > using more accurate watchlists).
> > -----Original Message-----
> > From: Jayson [mailto:jcasavant@x...]
> > Sent: Tuesday, December 30, 2003 9:52 AM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: RE: [amibroker] The demise of 23 systems...
> >
> >
> > Chuck,
> > does this suggest that all of your systems were simply curve
> fitted to
> > the current nas100? Or perhaps simply over optimized?
> >
> > Regards,
> > Jayson
> > -----Original Message-----
> > From: Chuck Rademacher [mailto:chuck_rademacher@x]
> > Sent: Tuesday, December 30, 2003 7:05 AM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: [amibroker] The demise of 23 systems...
> >
> >
> > I thought that I would share the following with you. Perhaps
> you can
> > see how the results of my research may affect your own systems?
> >
> > I had what I thought to be 23 "good" to "very good" systems
> that I
> > backtested against a current NASDAQ 100 watchlist. After
spending
> two days
> > updating symbols and accounting for de-listed stocks, I came up
with
> nine
> > separate watchlists, as at January 1 each year between 1995 and
> 2003.
> > Thanks to some of you, I modified my AFL to make sure that I used
> the
> > correct watchlist depending on the date. The results... not one
of
> my 23
> > NASDAQ 100 systems now backtest with acceptable performance. Many
> went from
> > nice, positive returns with low drawdowns to negative returns and
> almost
> > total loss of capital.
> >
> > What does it all mean? IMO, working with only a current
> NASDAQ 100
> > watchlist can give you a real sense of false security. Not only
> are
> > de-listed stocks not in the current list, but many of those
> de-listed stocks
> > had lacklustre performance before their demise.
> >
> > I encourage you to think about the impact of using only a
> current
> > watchlist, regardless of how that list is constructed, for
> backtesting
> > purposes. I see it all the time. System designers create
> watchlists of
> > high-beta or high-yield or low P/E or whatever stocks based on
> current
> > information to backtest a system starting perhaps five years
ago.
> Think
> > about it!
> >
> >
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> >
> >
> >
> --------------------------------------------------
> --------------------------
> > Yahoo! Groups Links
> >
> > a.. To visit your group on the web, go to:
> > http://groups.yahoo.com/group/amibroker/
> >
> > b.. To unsubscribe from this group, send an email to:
> > amibroker-unsubscribe@xxxxxxxxxxxxxxx
> >
> > c.. Your use of Yahoo! Groups is subject to the Yahoo! Terms
> of
> > Service.
> >
> >
> >
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> >
> >
> >
> >
> --------------------------------------------------
> --------------------------
> > Yahoo! Groups Links
> >
> > a.. To visit your group on the web, go to:
> > http://groups.yahoo.com/group/amibroker/
> >
> > b.. To unsubscribe from this group, send an email to:
> > amibroker-unsubscribe@xxxxxxxxxxxxxxx
> >
> > c.. Your use of Yahoo! Groups is subject to the Yahoo! Terms
> of
> > Service.
> >
> >
> >
> >
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> >
> >
> >
> --------------------------------------------------
> --------------------------
> > --
> > Yahoo! Groups Links
> >
> > a.. To visit your group on the web, go to:
> > http://groups.yahoo.com/group/amibroker/
> >
> > b.. To unsubscribe from this group, send an email to:
> > amibroker-unsubscribe@xxxxxxxxxxxxxxx
> >
> > c.. Your use of Yahoo! Groups is subject to the Yahoo! Terms
of
> Service.
> >
> >
> >
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> >
> >
> >
> --------------------------------------------------
> --------------------------
> > --
> > Yahoo! Groups Links
> >
> > a.. To visit your group on the web, go to:
> > http://groups.yahoo.com/group/amibroker/
> >
> > b.. To unsubscribe from this group, send an email to:
> > amibroker-unsubscribe@xxxxxxxxxxxxxxx
> >
> > c.. Your use of Yahoo! Groups is subject to the Yahoo! Terms
of
> Service.
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/GHeqlB/TM
---------------------------------------------------------------------~->
Yahoo! Groups Links
To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|