|
PureBytes Links
Trading Reference Links
|
Ok, I think I understand now. That also says to me that the systems
that you develop (which pass your criteria) have no problem being
profitable on N100 or SP500 or any other group. I've had a real tough
time achieving that. Systems that look great when entire universe is
concerned don't look as good when the universe is reduced to N100,
etc. One of these days I gotta go back and re-read all those posts
you made, because all of it was too much to grasp at that time. But
anyway, thanks for the explanation.
And happy new year everyone.
Jitu
--- In amibroker@xxxxxxxxxxxxxxx, "quanttrader714"
<quanttrader714@xxxx> wrote:
> Hi Jitu,
>
> --- In amibroker@xxxxxxxxxxxxxxx, "jtelang" <jtelang@xxxx> wrote:
> > Mark,
> >
> > I read your last two posts twice, and am having a tough time
> > understanding what you're saying... You said following in your
> first
> > post -
> >
> > > Then backtest on a random sample of issues drawn from the
> > > population of all stocks trading at the *start* of the period
> > > you're backtesting on. And oh, by the way, don't use *any* of
> > > this data to develop your system(s) because it theoretically
> > > didn't exist when you created them.
> >
> > What do you mean by "don't use any of this data..."? And what do
> you
> > mean it didn't exist when you created them? What is "them"?
>
> Here I was talking about eliminating bias by not using data to
create
> a system that theoretically was not available when the system would
> have been created.
>
> > And in this last post -
> >
> > > benchmark for *select* stocks from the population of stocks
> trading
> > at
> > > the start of the analysis period. I say select because, as you
> > know,
> > > N100 stocks are limited to non-financial companies listed on
> > NASDAQ.
>
> Here I was only saying how I'd *quantify* the N100 list "success
> story." For example, compare 98-03 buy and hold stats for *current*
> N100 stocks to the 98-03 buy and hold stats for the N100 eligible
> stocks *of 98*.
>
> > If the point is to limit your trading to N100 stocks, what's the
> > point in making sure that the backtest *also* works equally well
on
> > some other (non-N100) select stocks? No doubt, if you can make
both
> > happen (i.e. make it work equally well on random stocks as well
as
> > N100 stocks), then it's obviously great. But then you're
> effectively
> > eliminating any system that might work (for whatever reason) only
> > when it comes to N100 stocks, but not when entire universe is
> > considered. Are you saying such a beast can't exist? I, for one,
> > might lean towards agreeing with it, especially given Chuck's
> > comments as well as my own findings, but I hardly think one can
> > *conclusively* say so. Besides, if one really created accurate
N100
> > lists for past data, and develops a system that seems to trade
them
> > well in the past, should it be tossed out?
> >
> > I guess I'm trying to understand if your basic argument is -- any
> > system that backtests only using a group of stocks such as N100,
no
> > matter how accurate the data and lists were, are not something
one
> > should trade, because they were not backtested on non-N100
stocks.
> > Wasn't entirely clear if that's what you're saying.
>
> My basic point is that IMO, it's much easier and safer to trade the
> current N100 stocks with simple robust systems.
>
>
> Mark
>
> > TIA,
> >
> > Jitu
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "quanttrader714"
> > <quanttrader714@xxxx> wrote:
> > > Chuck,
> > >
> > > I've been following this thread and first let me thank you for
> > > sharing. It's always good to see a reminder that trading isn't
> > easy,
> > > especially from someone with your experience. This is a
complex
> > topic
> > > with many moving parts and I think that different people have
> > > different things in mind when discussing it (myself included,
> lol).
> > > You mentioned the current N100 list "success story" and I'd
> quantify
> > > this by looking at a benchmark for measuring it that is
> *unaffected
> > > by distortion from a system* (such as buy and hold profit) vs
the
> > same
> > > benchmark for *select* stocks from the population of stocks
> trading
> > at
> > > the start of the analysis period. I say select because, as you
> > know,
> > > N100 stocks are limited to non-financial companies listed on
> > NASDAQ.
> > > Then there's the broader issue of survivorship bias which I'd
also
> > > quantify without system distortion but can be avoided by acting
> only
> > > with information known at the time. What I *think* you did was
> try
> > to
> > > simulate the past with a one-frame-a-year rolling window of the
> > > N100 for your N100 systems (whatever they are). I'm not sure
what
> > > your results show about the N100 list success story or
> survivorship
> > > bias because of all the confounding variables that what I think
> > you've
> > > done introduces. But what I personally take away is another
> > argument
> > > for trading robust systems.
> > >
> > > Mark
> > >
> > > --- In amibroker@xxxxxxxxxxxxxxx, "Chuck Rademacher"
> > > <chuck_rademacher@x> wrote:
> > > > Jayson,
> > > >
> > > > Where do I begin? First, let me put your mind at ease and
> tell
> > you
> > > that I
> > > > am successfully trading numerous systems against the entire
> > > universe of
> > > > stocks. Suffice to say that I have a 40-year track record
> > trading
> > > my own
> > > > as well as other people's money. As I mentioned in my
earlier
> > > email, I
> > > > became frustrated by the fact that many people in this group
> were
> > > getting
> > > > such good backtested results from systems applied to the N100.
> > > >
> > > > My N100 systems being discussed performed very well in the
last
> > > year, not
> > > > poorly as you suggest. They performed very well for several
> > years
> > > against
> > > > the CURRENT N100 list. But, the CURRENT N100 list is a
> success
> > > story for
> > > > those 100 stocks.
> > > >
> > > > It is my belief that selecting and ranking based on only 100
> > stocks
> > > simply
> > > > won't yield enough qualified buy signals to give good returns
> > > without large
> > > > drawdowns. By the time I add liquidity and price filtering,
> > there
> > > may only
> > > > be one or two signals at each swing of the overall market.
> That
> > > simply
> > > > doesn't give me an acceptable level of diversfication. Due
to
> > the
> > > size of
> > > > the funds that I manage, I need to get off a fair amount of
> > dollars
> > > in a
> > > > fair number of positions in each system in order to play the
> game.
> > > >
> > > > I'll probably just go back to developing and trading systems
> that
> > > look at
> > > > the entire universe, rather than the N100. However, I will
> > suffer
> > > every
> > > > time I hear of someone seeing 2,500% annual returns in
> backtesting
> > > some new
> > > > system on the N100.
> > > > -----Original Message-----
> > > > From: Jayson [mailto:jcasavant@x...]
> > > > Sent: Wednesday, December 31, 2003 3:10 AM
> > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > Subject: RE: [amibroker] The demise of 23 systems...
> > > >
> > > >
> > > > Chuck,
> > > > Thanks for the detailed reply. You certainly have more
> > experience
> > > system
> > > > testing then I but it just seems that if a system "Works" it
> > should
> > > work on
> > > > any given data set at any given time. By this I mean that
there
> is
> > > really
> > > > nothing magical about the NAS100 or the SP500, they are just
> > stocks
> > > chosen,
> > > > for a variety of reasons, to be components of an index. If
the
> > > system looks
> > > > at market timing components then a stock will either
pass/fail
> > based
> > > on that
> > > > criteria. The same will be true for any additional components
> in
> > the
> > > system.
> > > > Why would stock "A" perform favorably while stock "B" does
not,
> > > after all it
> > > > passed the same (series) of tests.
> > > >
> > > > Time is also a component of testing that tends to confuse
me.
> > If a
> > > system
> > > > does poorly over the last 12 months why would you want to test
> it
> > > over a
> > > > longer period? True the system may produce favorable results
if
> > you
> > > had run
> > > > it for 10 years but if it is failing for the last year do you
> > really
> > > want to
> > > > be following it? Wouldn't you instead want to be looking for a
> > > system that
> > > > is behaving better today? If your system begins to fail
> wouldn't
> > you
> > > want to
> > > > stop trading it until such time as it begins behaving again?
> You
> > may
> > > miss
> > > > out on some winning trades but have you not saved your self
some
> > > losers in
> > > > the process? I have seen posts over the years that indicate
> some
> > are
> > > trying
> > > > to use MA's of the equity line as a filter to determine
> adherence
> > to
> > > a given
> > > > system. On the surface this seems to make sense to me. Have
you
> > > done any
> > > > work in this area?
> > > >
> > > > In answer 4 you state that the system will not optimize on
the
> > > revised
> > > > data. Does this not indicate that the system only worked on
the
> > > stocks
> > > > originally tested and therefore its success/failure was
actually
> > > random?
> > > > Pure luck based on the trades taken in the origininal data?
> > > >
> > > > Over the years I have tested various systems, some simple,
> some
> > > complex,
> > > > and have found the process frustrating. I can build a system
> that
> > > trades
> > > > well on a given universe of stocks. It will trade well on the
> the
> > > timeframe
> > > > I built the system on but when tested over a different time
> frame
> > or
> > > on a
> > > > different basket will invariably fall apart. It seems to me
> there
> > is
> > > > certainly an element of luck involved. Perhaps it is a coup[le
> of
> > > stocks in
> > > > the universe the have steadily produced, take them out
and ...no
> > > joy.
> > > > Perhaps a large percentage in the group had a stellar couple
of
> > > years. Test
> > > > out side that run up and again...no joy
> > > >
> > > > In your follow up you state that you have had little luck
> with
> > any
> > > system
> > > > testing less than 1,000 stocks. Do your systems take
> significantly
> > > more
> > > > trades when tested against a large group or are you simply
> trading
> > > stocks
> > > > meeting higher standards based on some type of ranking? If the
> > > latter than
> > > > perhaps stock selection is key. There certainly is a large
> group
> > of
> > > > believers in this theory over at IBD and at QP/HGS.
> > > >
> > > > I am sorry to hear your story as I am certain you have
> invested
> > > lots of
> > > > sweat equity into the development of your many systems. I
> > sincerely
> > > hope
> > > > that next year at this time you can share with us your success
> > > story.
> > > >
> > > >
> > > > Regards,
> > > > Jayson
> > > > -----Original Message-----
> > > > From: Chuck Rademacher [mailto:chuck_rademacher@x]
> > > > Sent: Tuesday, December 30, 2003 12:54 PM
> > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > Subject: RE: [amibroker] The demise of 23 systems...
> > > >
> > > >
> > > > Jayson,
> > > >
> > > > I can tell you, but probably not convince you, that the
> systems
> > > were not
> > > > over optimised. However, let me tell you the following so
you
> > can
> > > decide
> > > > for yourself:
> > > >
> > > > 1. All 23 of these systems were long only.
> > > >
> > > > 2. Each system had between 3 and 6 parameters, including
two
> > for
> > > market
> > > > timing.
> > > >
> > > > 3. The performance of these systems did not fall off a
cliff
> if
> > > any or
> > > > all of those parameters deviated in either direction.
> > > >
> > > > 4. Let's say that they were over optimised. The systems
> won't
> > > optimise
> > > > with the "revised" data.
> > > >
> > > >
> > > > I have to tell you a couple more things:
> > > >
> > > > 1. I tend to use exactly the same two-parameter market
timing
> > > methodology
> > > > (including parameter values) for more than 120 systems.
> > > >
> > > > 2. I find that the more stocks I pump into a system, the
> more
> > it
> > > has to
> > > > select from and the better the performance.
> > > >
> > > > 3. I could never get any systems to work to my satisfaction
> on
> > > less than
> > > > 1,000 stocks.
> > > >
> > > > 4. I certainly could never get any system to work to my
> > > satisfaction on
> > > > the NASDAQ 100.
> > > >
> > > > 5. All of my systems work (40+ years) was previously done
> > > outside of
> > > > AmiBroker where my database knows which stocks belong to
which
> > index
> > > at any
> > > > given minute in time.
> > > >
> > > > 6. I became frustrated with reports of numerous AmiBroker
> users
> > > having
> > > > success with NASDAQ 100 trading systems.
> > > >
> > > > 7. I created a (current) NASDAQ 100 watchlist and a bunch
of
> > > systems that
> > > > appeared to work.
> > > >
> > > > 8. I'm actually trading 11 of the 23 systems that I
> > mentioned.
> > > Well, to
> > > > be honest, I was trading them until yesterday.
> > > >
> > > > 9. Once I tried the more accurate method of determining
which
> > > stocks were
> > > > NASDAQ 100 constituents in AmiBroker, these systems fell
apart.
> > > >
> > > > That's my story... back to the drawing board (unless I want
to
> > > ignore
> > > > using more accurate watchlists).
> > > > -----Original Message-----
> > > > From: Jayson [mailto:jcasavant@x...]
> > > > Sent: Tuesday, December 30, 2003 9:52 AM
> > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > Subject: RE: [amibroker] The demise of 23 systems...
> > > >
> > > >
> > > > Chuck,
> > > > does this suggest that all of your systems were simply
curve
> > > fitted to
> > > > the current nas100? Or perhaps simply over optimized?
> > > >
> > > > Regards,
> > > > Jayson
> > > > -----Original Message-----
> > > > From: Chuck Rademacher [mailto:chuck_rademacher@x]
> > > > Sent: Tuesday, December 30, 2003 7:05 AM
> > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > Subject: [amibroker] The demise of 23 systems...
> > > >
> > > >
> > > > I thought that I would share the following with you.
> Perhaps
> > > you can
> > > > see how the results of my research may affect your own
systems?
> > > >
> > > > I had what I thought to be 23 "good" to "very good"
systems
> > > that I
> > > > backtested against a current NASDAQ 100 watchlist. After
> > spending
> > > two days
> > > > updating symbols and accounting for de-listed stocks, I came
up
> > with
> > > nine
> > > > separate watchlists, as at January 1 each year between 1995
and
> > > 2003.
> > > > Thanks to some of you, I modified my AFL to make sure that I
> used
> > > the
> > > > correct watchlist depending on the date. The results... not
> one
> > of
> > > my 23
> > > > NASDAQ 100 systems now backtest with acceptable performance.
> Many
> > > went from
> > > > nice, positive returns with low drawdowns to negative returns
> and
> > > almost
> > > > total loss of capital.
> > > >
> > > > What does it all mean? IMO, working with only a current
> > > NASDAQ 100
> > > > watchlist can give you a real sense of false security. Not
> only
> > > are
> > > > de-listed stocks not in the current list, but many of those
> > > de-listed stocks
> > > > had lacklustre performance before their demise.
> > > >
> > > > I encourage you to think about the impact of using only a
> > > current
> > > > watchlist, regardless of how that list is constructed, for
> > > backtesting
> > > > purposes. I see it all the time. System designers create
> > > watchlists of
> > > > high-beta or high-yield or low P/E or whatever stocks based on
> > > current
> > > > information to backtest a system starting perhaps five years
> > ago.
> > > Think
> > > > about it!
> > > >
> > > >
> > > > Send BUG REPORTS to bugs@xxxx
> > > > Send SUGGESTIONS to suggest@xxxx
> > > > -----------------------------------------
> > > > Post AmiQuote-related messages ONLY to:
> > amiquote@xxxxxxxxxxxxxxx
> > > > (Web page:
http://groups.yahoo.com/group/amiquote/messages/)
> > > > --------------------------------------------
> > > > Check group FAQ at:
> > > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > >
> > > >
> > > >
> > > >
> > > --------------------------------------------------
> > > --------------------------
> > > > Yahoo! Groups Links
> > > >
> > > > a.. To visit your group on the web, go to:
> > > > http://groups.yahoo.com/group/amibroker/
> > > >
> > > > b.. To unsubscribe from this group, send an email to:
> > > > amibroker-unsubscribe@xxxxxxxxxxxxxxx
> > > >
> > > > c.. Your use of Yahoo! Groups is subject to the Yahoo!
> Terms
> > > of
> > > > Service.
> > > >
> > > >
> > > >
> > > > Send BUG REPORTS to bugs@xxxx
> > > > Send SUGGESTIONS to suggest@xxxx
> > > > -----------------------------------------
> > > > Post AmiQuote-related messages ONLY to:
> > amiquote@xxxxxxxxxxxxxxx
> > > > (Web page:
http://groups.yahoo.com/group/amiquote/messages/)
> > > > --------------------------------------------
> > > > Check group FAQ at:
> > > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > >
> > > >
> > > >
> > > >
> > > >
> > > --------------------------------------------------
> > > --------------------------
> > > > Yahoo! Groups Links
> > > >
> > > > a.. To visit your group on the web, go to:
> > > > http://groups.yahoo.com/group/amibroker/
> > > >
> > > > b.. To unsubscribe from this group, send an email to:
> > > > amibroker-unsubscribe@xxxxxxxxxxxxxxx
> > > >
> > > > c.. Your use of Yahoo! Groups is subject to the Yahoo!
> Terms
> > > of
> > > > Service.
> > > >
> > > >
> > > >
> > > >
> > > > Send BUG REPORTS to bugs@xxxx
> > > > Send SUGGESTIONS to suggest@xxxx
> > > > -----------------------------------------
> > > > Post AmiQuote-related messages ONLY to: am
> iquote@xxxxxxxxxxxxxxx
> > > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > > --------------------------------------------
> > > > Check group FAQ at:
> > > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > >
> > > >
> > > >
> > > >
> > > --------------------------------------------------
> > > --------------------------
> > > > --
> > > > Yahoo! Groups Links
> > > >
> > > > a.. To visit your group on the web, go to:
> > > > http://groups.yahoo.com/group/amibroker/
> > > >
> > > > b.. To unsubscribe from this group, send an email to:
> > > > amibroker-unsubscribe@xxxxxxxxxxxxxxx
> > > >
> > > > c.. Your use of Yahoo! Groups is subject to the Yahoo!
> Terms
> > of
> > > Service.
> > > >
> > > >
> > > >
> > > > Send BUG REPORTS to bugs@xxxx
> > > > Send SUGGESTIONS to suggest@xxxx
> > > > -----------------------------------------
> > > > Post AmiQuote-related messages ONLY to: am
> iquote@xxxxxxxxxxxxxxx
> > > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > > --------------------------------------------
> > > > Check group FAQ at:
> > > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > >
> > > >
> > > >
> > > >
> > > --------------------------------------------------
> > > --------------------------
> > > > --
> > > > Yahoo! Groups Links
> > > >
> > > > a.. To visit your group on the web, go to:
> > > > http://groups.yahoo.com/group/amibroker/
> > > >
> > > > b.. To unsubscribe from this group, send an email to:
> > > > amibroker-unsubscribe@xxxxxxxxxxxxxxx
> > > >
> > > > c.. Your use of Yahoo! Groups is subject to the Yahoo!
> Terms
> > of
> > > Service.
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Yahoo! Groups Links
To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|