|
PureBytes Links
Trading Reference Links
|
Chuck,
I've been following this thread and first let me thank you for
sharing. It's always good to see a reminder that trading isn't easy,
especially from someone with your experience. This is a complex topic
with many moving parts and I think that different people have
different things in mind when discussing it (myself included, lol).
You mentioned the current N100 list "success story" and I'd quantify
this by looking at a benchmark for measuring it that is *unaffected
by distortion from a system* (such as buy and hold profit) vs the same
benchmark for *select* stocks from the population of stocks trading at
the start of the analysis period. I say select because, as you know,
N100 stocks are limited to non-financial companies listed on NASDAQ.
Then there's the broader issue of survivorship bias which I'd also
quantify without system distortion but can be avoided by acting only
with information known at the time. What I *think* you did was try to
simulate the past with a one-frame-a-year rolling window of the
N100 for your N100 systems (whatever they are). I'm not sure what
your results show about the N100 list success story or survivorship
bias because of all the confounding variables that what I think you've
done introduces. But what I personally take away is another argument
for trading robust systems.
Mark
--- In amibroker@xxxxxxxxxxxxxxx, "Chuck Rademacher"
<chuck_rademacher@x> wrote:
> Jayson,
>
> Where do I begin? First, let me put your mind at ease and tell you
that I
> am successfully trading numerous systems against the entire
universe of
> stocks. Suffice to say that I have a 40-year track record trading
my own
> as well as other people's money. As I mentioned in my earlier
email, I
> became frustrated by the fact that many people in this group were
getting
> such good backtested results from systems applied to the N100.
>
> My N100 systems being discussed performed very well in the last
year, not
> poorly as you suggest. They performed very well for several years
against
> the CURRENT N100 list. But, the CURRENT N100 list is a success
story for
> those 100 stocks.
>
> It is my belief that selecting and ranking based on only 100 stocks
simply
> won't yield enough qualified buy signals to give good returns
without large
> drawdowns. By the time I add liquidity and price filtering, there
may only
> be one or two signals at each swing of the overall market. That
simply
> doesn't give me an acceptable level of diversfication. Due to the
size of
> the funds that I manage, I need to get off a fair amount of dollars
in a
> fair number of positions in each system in order to play the game.
>
> I'll probably just go back to developing and trading systems that
look at
> the entire universe, rather than the N100. However, I will suffer
every
> time I hear of someone seeing 2,500% annual returns in backtesting
some new
> system on the N100.
> -----Original Message-----
> From: Jayson [mailto:jcasavant@x...]
> Sent: Wednesday, December 31, 2003 3:10 AM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: RE: [amibroker] The demise of 23 systems...
>
>
> Chuck,
> Thanks for the detailed reply. You certainly have more experience
system
> testing then I but it just seems that if a system "Works" it should
work on
> any given data set at any given time. By this I mean that there is
really
> nothing magical about the NAS100 or the SP500, they are just stocks
chosen,
> for a variety of reasons, to be components of an index. If the
system looks
> at market timing components then a stock will either pass/fail based
on that
> criteria. The same will be true for any additional components in the
system.
> Why would stock "A" perform favorably while stock "B" does not,
after all it
> passed the same (series) of tests.
>
> Time is also a component of testing that tends to confuse me. If a
system
> does poorly over the last 12 months why would you want to test it
over a
> longer period? True the system may produce favorable results if you
had run
> it for 10 years but if it is failing for the last year do you really
want to
> be following it? Wouldn't you instead want to be looking for a
system that
> is behaving better today? If your system begins to fail wouldn't you
want to
> stop trading it until such time as it begins behaving again? You may
miss
> out on some winning trades but have you not saved your self some
losers in
> the process? I have seen posts over the years that indicate some are
trying
> to use MA's of the equity line as a filter to determine adherence to
a given
> system. On the surface this seems to make sense to me. Have you
done any
> work in this area?
>
> In answer 4 you state that the system will not optimize on the
revised
> data. Does this not indicate that the system only worked on the
stocks
> originally tested and therefore its success/failure was actually
random?
> Pure luck based on the trades taken in the origininal data?
>
> Over the years I have tested various systems, some simple, some
complex,
> and have found the process frustrating. I can build a system that
trades
> well on a given universe of stocks. It will trade well on the the
timeframe
> I built the system on but when tested over a different time frame or
on a
> different basket will invariably fall apart. It seems to me there is
> certainly an element of luck involved. Perhaps it is a coup[le of
stocks in
> the universe the have steadily produced, take them out and ...no
joy.
> Perhaps a large percentage in the group had a stellar couple of
years. Test
> out side that run up and again...no joy
>
> In your follow up you state that you have had little luck with any
system
> testing less than 1,000 stocks. Do your systems take significantly
more
> trades when tested against a large group or are you simply trading
stocks
> meeting higher standards based on some type of ranking? If the
latter than
> perhaps stock selection is key. There certainly is a large group of
> believers in this theory over at IBD and at QP/HGS.
>
> I am sorry to hear your story as I am certain you have invested
lots of
> sweat equity into the development of your many systems. I sincerely
hope
> that next year at this time you can share with us your success
story.
>
>
> Regards,
> Jayson
> -----Original Message-----
> From: Chuck Rademacher [mailto:chuck_rademacher@x...]
> Sent: Tuesday, December 30, 2003 12:54 PM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: RE: [amibroker] The demise of 23 systems...
>
>
> Jayson,
>
> I can tell you, but probably not convince you, that the systems
were not
> over optimised. However, let me tell you the following so you can
decide
> for yourself:
>
> 1. All 23 of these systems were long only.
>
> 2. Each system had between 3 and 6 parameters, including two for
market
> timing.
>
> 3. The performance of these systems did not fall off a cliff if
any or
> all of those parameters deviated in either direction.
>
> 4. Let's say that they were over optimised. The systems won't
optimise
> with the "revised" data.
>
>
> I have to tell you a couple more things:
>
> 1. I tend to use exactly the same two-parameter market timing
methodology
> (including parameter values) for more than 120 systems.
>
> 2. I find that the more stocks I pump into a system, the more it
has to
> select from and the better the performance.
>
> 3. I could never get any systems to work to my satisfaction on
less than
> 1,000 stocks.
>
> 4. I certainly could never get any system to work to my
satisfaction on
> the NASDAQ 100.
>
> 5. All of my systems work (40+ years) was previously done
outside of
> AmiBroker where my database knows which stocks belong to which index
at any
> given minute in time.
>
> 6. I became frustrated with reports of numerous AmiBroker users
having
> success with NASDAQ 100 trading systems.
>
> 7. I created a (current) NASDAQ 100 watchlist and a bunch of
systems that
> appeared to work.
>
> 8. I'm actually trading 11 of the 23 systems that I mentioned.
Well, to
> be honest, I was trading them until yesterday.
>
> 9. Once I tried the more accurate method of determining which
stocks were
> NASDAQ 100 constituents in AmiBroker, these systems fell apart.
>
> That's my story... back to the drawing board (unless I want to
ignore
> using more accurate watchlists).
> -----Original Message-----
> From: Jayson [mailto:jcasavant@x...]
> Sent: Tuesday, December 30, 2003 9:52 AM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: RE: [amibroker] The demise of 23 systems...
>
>
> Chuck,
> does this suggest that all of your systems were simply curve
fitted to
> the current nas100? Or perhaps simply over optimized?
>
> Regards,
> Jayson
> -----Original Message-----
> From: Chuck Rademacher [mailto:chuck_rademacher@x...]
> Sent: Tuesday, December 30, 2003 7:05 AM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: [amibroker] The demise of 23 systems...
>
>
> I thought that I would share the following with you. Perhaps
you can
> see how the results of my research may affect your own systems?
>
> I had what I thought to be 23 "good" to "very good" systems
that I
> backtested against a current NASDAQ 100 watchlist. After spending
two days
> updating symbols and accounting for de-listed stocks, I came up with
nine
> separate watchlists, as at January 1 each year between 1995 and
2003.
> Thanks to some of you, I modified my AFL to make sure that I used
the
> correct watchlist depending on the date. The results... not one of
my 23
> NASDAQ 100 systems now backtest with acceptable performance. Many
went from
> nice, positive returns with low drawdowns to negative returns and
almost
> total loss of capital.
>
> What does it all mean? IMO, working with only a current
NASDAQ 100
> watchlist can give you a real sense of false security. Not only
are
> de-listed stocks not in the current list, but many of those
de-listed stocks
> had lacklustre performance before their demise.
>
> I encourage you to think about the impact of using only a
current
> watchlist, regardless of how that list is constructed, for
backtesting
> purposes. I see it all the time. System designers create
watchlists of
> high-beta or high-yield or low P/E or whatever stocks based on
current
> information to backtest a system starting perhaps five years ago.
Think
> about it!
>
>
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
>
>
>
>
--------------------------------------------------
--------------------------
> Yahoo! Groups Links
>
> a.. To visit your group on the web, go to:
> http://groups.yahoo.com/group/amibroker/
>
> b.. To unsubscribe from this group, send an email to:
> amibroker-unsubscribe@xxxxxxxxxxxxxxx
>
> c.. Your use of Yahoo! Groups is subject to the Yahoo! Terms
of
> Service.
>
>
>
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
>
>
>
>
>
--------------------------------------------------
--------------------------
> Yahoo! Groups Links
>
> a.. To visit your group on the web, go to:
> http://groups.yahoo.com/group/amibroker/
>
> b.. To unsubscribe from this group, send an email to:
> amibroker-unsubscribe@xxxxxxxxxxxxxxx
>
> c.. Your use of Yahoo! Groups is subject to the Yahoo! Terms
of
> Service.
>
>
>
>
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
>
>
>
>
--------------------------------------------------
--------------------------
> --
> Yahoo! Groups Links
>
> a.. To visit your group on the web, go to:
> http://groups.yahoo.com/group/amibroker/
>
> b.. To unsubscribe from this group, send an email to:
> amibroker-unsubscribe@xxxxxxxxxxxxxxx
>
> c.. Your use of Yahoo! Groups is subject to the Yahoo! Terms of
Service.
>
>
>
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
>
>
>
>
--------------------------------------------------
--------------------------
> --
> Yahoo! Groups Links
>
> a.. To visit your group on the web, go to:
> http://groups.yahoo.com/group/amibroker/
>
> b.. To unsubscribe from this group, send an email to:
> amibroker-unsubscribe@xxxxxxxxxxxxxxx
>
> c.. Your use of Yahoo! Groups is subject to the Yahoo! Terms of
Service.
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Yahoo! Groups Links
To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|