|
PureBytes Links
Trading Reference Links
|
Al,
Your plan to use 2 random watchlists may give a type of "out of
sample" verification, but it will only be so for PART of a system if
that system uses market timing signals. The multiple random
watchlist approach will give "some" OOS verification for stock
selection filters and positionscore ranking methods.
If the system is based on a market timing signal to go Long and
Short, retesting on the second watchlist will do NEXT TO NOTHING to
verify the soundness (or lack thereof) of the timing signal.
b
--- In amibroker@xxxxxxxxxxxxxxx, "Al Venosa" <advenosa@xxxx> wrote:
> Pal:
>
> Sorry, but I changed the title to something that can be searched
on. Anyway, I write this because I read with interest an idea that I
believe Chuck expressed this morning in an unrelated post. Suppose
your watchlist is composed of, say, the Nasdaq 100 (N100) (it could
be anything; this is just an example). Divide the N100 stocks into 2
50-stock watchlists by randomly selecting 50 stocks from the N100
for each watchlist. Then, do your development/optimization on one of
the 2 watchlists for 1999 to Dec 2003 (present time). Then, use the
optimized parameters plus the OTHER watchlist for your "walk-
forward" testing. This way, you use the entire date range but you
test the system's validity or robustness on another watchlist that
has never seen the parameters or the system at all. If the system
performs as well or better on the second watchlist as it did on the
one used for the optimum parameters, then you may conclude you have
a tradeable system. I'd like to hear others' opinions as to the
validity of this approach. Any thoughts?
>
> AV
> ----- Original Message -----
> From: palsanand
> To: amibroker@xxxxxxxxxxxxxxx
> Sent: Saturday, December 13, 2003 5:38 PM
> Subject: [amibroker] Re: question/ help
>
>
> I would point you to the following article:
>
> Walk Forward Tested System Is Better than the Best
> Optimized One - Larry Williams
>
> It looks like the history of technical analysis has been largely
> influenced by optimization. That is, we studied the past, found
> something that looked significant, then optimized rules and
> procedures to trade the observation in the future.
>
> Sometimes that has worked. Often it has not. That's our dilemma.
What
> are we to do? In the past, we answered these questions by doing
more
> optimization, more curve fitting. Indeed, we treated historical
data
> like prisoners of war. Our thesis was, if you beat them often
enough
> they would reveal anything. Which is true, but you want them to
> reveal everything, not anything.
>
> This brings me to one point. I think we will all make much more
> headway with system development by spending less time on
optimization
> and more time on walking systems and procedures forward.
>
> If on a walk forward test, the system holds up, we probably have
> something. And for sure, what we have will be better than the
very
> best optimized system when it comes to real time trading. Hence,
> let's see what we can learn from each other about conducting
walk
> forward tests. Any ideas will be appreciated by all, I am
certain -
> L.W.
>
> It's certainly correct that if done properly, walk forward
testing
> has great value. For those of you not aware of walk forward
testing,
> it's first setting your system parameters and then testing the
> results in the future using those pre-set parameters without
benefit
> of additional or new optimization (re-optimization). Some people
> refer to that as "hypothetical real-time trading."
>
> However, walk forward testing can in fact be a trap if done
> incorrectly. That's because there's a problem in deciding what
pre-
> set algorithm or parameters to use prior to the so-called walk
> forward test. If we arrive at those parameters by an
optimization
> process, then we may be guilty of optimizing the walk forward
test
> without even realizing we have done that. Another pitfall, is
the
> great tendency to optimize the walk forward testing time period
> itself.
>
> Possibly the only way to do it correctly, is to first arrive at
a set
> of parameters and algorithm based on logic, experience, or sound
> trading principles that won't be subject to change. Then do a
walk
> forward with no attempt to improve results via re-optimization.
>
> A robust system is one that uses the least amount of parameters
and
> still has a good chance of being profitable in the future with
> atleast 30 trades/year to be statistically significant with the
in-
> sample testing period of atleast say, 15 years to minimize the
> possibility of chance or chance alone being responsible for the
> excellent results - Pal
>
> rgds, Pal
>
> --- In amibroker@xxxxxxxxxxxxxxx, "goldwing01" <GOLDWING01@xxxx>
> wrote:
> > Has any tried using databull for downloading data, it seems to
look
> > better than yahoo or a least relieving yourself from the
stress of
> > downloading problem.
> >
> > Next question, optimization, why do I need to optimize my data
and
> > when should I optimize my data
> >
> > Next question, I would like to have one screen to constantly
look
> at
> > ^IXIC and another to move as I point and click at other
symbols can
> > someone help with this please.
>
>
> Yahoo! Groups Sponsor
> ADVERTISEMENT
>
>
>
>
>
> Send BUG REPORTS to bugs@xxxx
> Send SUGGESTIONS to suggest@xxxx
> -----------------------------------------
> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
> (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> --------------------------------------------
> Check group FAQ at:
http://groups.yahoo.com/group/amibroker/files/groupfaq.html
>
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of
Service.
>
>
> ---
> Outgoing mail is certified Virus Free.
> Checked by AVG anti-virus system (http://www.grisoft.com).
> Version: 6.0.543 / Virus Database: 337 - Release Date: 11/21/2003
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|