|
PureBytes Links
Trading Reference Links
|
<FONT face=Arial color=#0000ff
size=2>Chuck,
Thanks
for the detailed reply. You certainly have more experience system testing then I
but it just seems that if a system "Works" it should work on any given data set
at any given time. By this I mean that there is really nothing magical about the
NAS100 or the SP500, they are just stocks chosen, for a variety of reasons,
to be components of an index. If the system looks at market timing
components then a stock will either pass/fail based on that criteria. The same
will be true for any additional components in the system. Why would stock "A"
perform favorably while stock "B" does not, after all it passed the same
(series) of tests.
<FONT face=Arial color=#0000ff
size=2>
Time
is also a component of testing that tends to confuse me. If a system does poorly
over the last 12 months why would you want to test it over a longer period? True
the system may produce favorable results if you had run it for 10 years but if
it is failing for the last year do you really want to be following it? Wouldn't
you instead want to be looking for a system that is behaving better today? If
your system begins to fail wouldn't you want to stop trading it until such time
as it begins behaving again? You may miss out on some winning trades but have
you not saved your self some losers in the process? I have seen posts over the
years that indicate some are trying to use MA's of the equity line as a filter
to determine adherence to a given system. On the surface this seems to make
sense to me. Have you done any work in this area?
<FONT face=Arial color=#0000ff
size=2>
In
answer 4 you state that the system will not optimize on the revised data. Does
this not indicate that the system only worked on the stocks originally tested
and therefore its success/failure was actually random? Pure luck based on the
trades taken in the origininal data?
<FONT face=Arial color=#0000ff
size=2>
<FONT
face=Arial color=#0000ff size=2>Over the years I have tested various systems,
some simple, some complex, and have found the process frustrating. I can build a
system that trades well on a given universe of stocks. It will trade well on the
the timeframe I built the system on but when tested over a different time frame
or on a different basket will invariably fall apart. It seems to me there is
certainly an element of luck involved. Perhaps it is a coup[le of stocks in the
universe the have steadily produced, take them out and ...no joy. Perhaps a
large percentage in the group had a stellar couple of years. Test out side that
run up and again...no joy
<FONT face=Arial color=#0000ff
size=2>
In
your follow up you state that you have had little luck with any system testing
less than 1,000 stocks. Do your systems take significantly more trades when
tested against a large group or are you simply trading stocks meeting higher
standards based on some type of ranking? If the latter than perhaps stock
selection is key. There certainly is a large group of believers in this theory
over at IBD and at QP/HGS.
<FONT face=Arial color=#0000ff
size=2>
I am
sorry to hear your story as I am certain you have invested lots of sweat equity
into the development of your many systems. I sincerely hope that next year at
this time you can share with us your success story.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>
Regards,
Jayson
<FONT face=Tahoma
size=2>-----Original Message-----From: Chuck Rademacher
[mailto:chuck_rademacher@xxxxxxxxxx]Sent: Tuesday, December 30, 2003
12:54 PMTo: amibroker@xxxxxxxxxxxxxxxSubject: RE:
[amibroker] The demise of 23 systems...
<FONT face=Arial color=#0000ff
size=2>Jayson,
<FONT face=Arial color=#0000ff
size=2>
I can
tell you, but probably not convince you, that the systems were not over
optimised. However, let me tell you the following so you can decide
for yourself:
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>1. All 23 of these systems were long only.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>2. Each system had between 3 and 6 parameters, including two for
market timing.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>3. The performance of these systems did not fall off a cliff if any
or all of those parameters deviated in either direction.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>4. Let's say that they were over optimised. The
systems won't optimise with the "revised" data.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>
I have
to tell you a couple more things:
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>1. I tend to use exactly the same two-parameter market timing
methodology (including parameter values) for more than 120
systems.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>2. I find that the more stocks I pump into a system, the more it
has to select from and the better the performance.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>3. I could never get any systems to work to my satisfaction on less
than 1,000 stocks.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>4. I certainly could never get any system to work to my
satisfaction on the NASDAQ 100.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>5. All of my systems work (40+ years) was previously done outside
of AmiBroker where my database knows which stocks belong to which
index at any given minute in time.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>6. I became frustrated with reports of numerous AmiBroker users
having success with NASDAQ 100 trading systems.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>7. I created a (current) NASDAQ 100 watchlist and a bunch of
systems that appeared to work.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>8. I'm actually trading 11 of the 23 systems that I
mentioned. Well, to be honest, I was trading them until
yesterday.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>9. Once I tried the more accurate method of determining which
stocks were NASDAQ 100 constituents in AmiBroker, these systems fell
apart.
<FONT face=Arial color=#0000ff
size=2>
That's
my story... back to the drawing board (unless I want to ignore using more
accurate watchlists).
<BLOCKQUOTE
>
<FONT face="Times New Roman"
size=2>-----Original Message-----From: Jayson
[mailto:jcasavant@xxxxxxxxxxx]Sent: Tuesday, December 30, 2003 9:52
AMTo: amibroker@xxxxxxxxxxxxxxxSubject: RE: [amibroker]
The demise of 23 systems...
<FONT face=Arial color=#0000ff
size=2>Chuck,
does
this suggest that all of your systems were simply curve fitted to the current
nas100? Or perhaps simply over optimized?
Regards,
Jayson
<FONT face=Tahoma
size=2>-----Original Message-----From: Chuck Rademacher
[mailto:chuck_rademacher@xxxxxxxxxx]Sent: Tuesday, December 30,
2003 7:05 AMTo: amibroker@xxxxxxxxxxxxxxxSubject:
[amibroker] The demise of 23 systems...
I
thought that I would share the following with you. Perhaps you can
see how the results of my research may affect your own
systems?
<FONT face=Arial color=#0000ff
size=2>
I
had what I thought to be 23 "good" to "very good" systems that I backtested
against a current NASDAQ 100 watchlist. After spending
two days updating symbols and accounting for de-listed stocks, I came up with
nine separate watchlists, as at January 1 each year between 1995 and
2003. Thanks to some of you, I modified my AFL to make sure that I
used the correct watchlist depending on the date. The results...
not one of my 23 NASDAQ 100 systems now backtest with acceptable
performance. Many went from nice, positive returns with low drawdowns to
negative returns and almost total loss of capital.
<FONT face=Arial color=#0000ff
size=2>
What
does it all mean? IMO, working with only a current NASDAQ 100
watchlist can give you a real sense of false security. Not only
are de-listed stocks not in the current list, but many of those de-listed
stocks had lacklustre performance before their demise.
<FONT face=Arial color=#0000ff
size=2>
I
encourage you to think about the impact of using only a current watchlist,
regardless of how that list is constructed, for backtesting
purposes. I see it all the time. System designers
create watchlists of high-beta or high-yield or low P/E or whatever stocks
based on current information to backtest a system starting perhaps five years
ago. Think about it!Send BUG
REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Yahoo! Groups Links
To visit your group on the web, go to:<A
href="">http://groups.yahoo.com/group/amibroker/
To unsubscribe from this group, send an email to:<A
href="">amibroker-unsubscribe@xxxxxxxxxxxxxxx
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Send BUG REPORTS to bugs@xxxxxxxxxxxxxSend
SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Yahoo! Groups Links
To visit your group on the web, go to:<A
href="">http://groups.yahoo.com/group/amibroker/
To unsubscribe from this group, send an email to:<A
href="">amibroker-unsubscribe@xxxxxxxxxxxxxxx
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Send BUG REPORTS to
bugs@xxxxxxxxxxxxxSend SUGGESTIONS to
suggest@xxxxxxxxxxxxx-----------------------------------------Post
AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: <A
href="">http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check
group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Yahoo! Groups Links
To visit your group on the web, go to:<A
href="">http://groups.yahoo.com/group/amibroker/
To unsubscribe from this group, send an email to:<A
href="">amibroker-unsubscribe@xxxxxxxxxxxxxxx
Your use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of Service.
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Yahoo! Groups Links
To visit your group on the web, go to:http://groups.yahoo.com/group/amibroker/
To unsubscribe from this group, send an email to:amibroker-unsubscribe@xxxxxxxxxxxxxxx
Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
|