|
PureBytes Links
Trading Reference Links
|
b,
That's a good point.
One of the things I've been mulling over related to all of this is the danger of losing the forest for the trees by forgetting that this process all starts with a good timing signal with which we'd like to find good stocks. Consequently, I'm staying away from doing any additional tuning to params of timing signal once I marry it to a selection methodology.
Ultimately, this all relates back to your point that assuming your due dilligence is complete on the market timing signal you can go forward with the process Al has described unheeded as it is only selection related.
Regards,
Garyb519b <b519b@xxxxxxxxx> wrote:
Al,Your plan to use 2 random watchlists may give a type of "out of sample" verification, but it will only be so for PART of a system if that system uses market timing signals. The multiple random watchlist approach will give "some" OOS verification for stock selection filters and positionscore ranking methods. If the system is based on a market timing signal to go Long and Short, retesting on the second watchlist will do NEXT TO NOTHING to verify the soundness (or lack thereof) of the timing signal. b--- In amibroker@xxxxxxxxxxxxxxx, "Al Venosa" <advenosa@xxxx> wrote:> Pal:> > Sorry, but I changed the title to something that can be searched on. Anyway, I write this because I read with interest an idea that I believe Chuck expressed this morning in an unrelated post. Suppose your
watchlist is composed of, say, the Nasdaq 100 (N100) (it could be anything; this is just an example). Divide the N100 stocks into 2 50-stock watchlists by randomly selecting 50 stocks from the N100 for each watchlist. Then, do your development/optimization on one of the 2 watchlists for 1999 to Dec 2003 (present time). Then, use the optimized parameters plus the OTHER watchlist for your "walk-forward" testing. This way, you use the entire date range but you test the system's validity or robustness on another watchlist that has never seen the parameters or the system at all. If the system performs as well or better on the second watchlist as it did on the one used for the optimum parameters, then you may conclude you have a tradeable system. I'd like to hear others' opinions as to the validity of this approach. Any thoughts?> > AV> ----- Original Message ----- > From: palsanand
> To: amibroker@xxxxxxxxxxxxxxx > Sent: Saturday, December 13, 2003 5:38 PM> Subject: [amibroker] Re: question/ help> > > I would point you to the following article:> > Walk Forward Tested System Is Better than the Best> Optimized One - Larry Williams> > It looks like the history of technical analysis has been largely > influenced by optimization. That is, we studied the past, found > something that looked significant, then optimized rules and > procedures to trade the observation in the future.> > Sometimes that has worked. Often it has not. That's our dilemma. What > are we to do? In the past, we answered these questions by doing more > optimization, more curve fitting. Indeed, we treated
historical data > like prisoners of war. Our thesis was, if you beat them often enough > they would reveal anything. Which is true, but you want them to > reveal everything, not anything.> > This brings me to one point. I think we will all make much more > headway with system development by spending less time on optimization > and more time on walking systems and procedures forward.> > If on a walk forward test, the system holds up, we probably have > something. And for sure, what we have will be better than the very > best optimized system when it comes to real time trading. Hence, > let's see what we can learn from each other about conducting walk > forward tests. Any ideas will be appreciated by all, I am certain - >
L.W.> > It's certainly correct that if done properly, walk forward testing > has great value. For those of you not aware of walk forward testing, > it's first setting your system parameters and then testing the > results in the future using those pre-set parameters without benefit > of additional or new optimization (re-optimization). Some people > refer to that as "hypothetical real-time trading."> > However, walk forward testing can in fact be a trap if done > incorrectly. That's because there's a problem in deciding what pre-> set algorithm or parameters to use prior to the so-called walk > forward test. If we arrive at those parameters by an optimization > process, then we may be guilty of optimizing the walk forward test
> without even realizing we have done that. Another pitfall, is the > great tendency to optimize the walk forward testing time period > itself.> > Possibly the only way to do it correctly, is to first arrive at a set > of parameters and algorithm based on logic, experience, or sound > trading principles that won't be subject to change. Then do a walk > forward with no attempt to improve results via re-optimization.> > A robust system is one that uses the least amount of parameters and > still has a good chance of being profitable in the future with > atleast 30 trades/year to be statistically significant with the in-> sample testing period of atleast say, 15 years to minimize the > possibility of chance or chance alone being
responsible for the > excellent results - Pal> > rgds, Pal> > --- In amibroker@xxxxxxxxxxxxxxx, "goldwing01" <GOLDWING01@xxxx> > wrote:> > Has any tried using databull for downloading data, it seems to look > > better than yahoo or a least relieving yourself from the stress of > > downloading problem.> > > > Next question, optimization, why do I need to optimize my data and > > when should I optimize my data > > > > Next question, I would like to have one screen to constantly look > at > > ^IXIC and another to move as I point and click at other symbols can > > someone help with this please.> >
> Yahoo! Groups Sponsor > ADVERTISEMENT> > > > > > Send BUG REPORTS to bugs@xxxx> Send SUGGESTIONS to suggest@xxxx> -----------------------------------------> Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx > (Web page: http://groups.yahoo.com/group/amiquote/messages/)> --------------------------------------------> Check group FAQ at: <A
href="">http://groups.yahoo.com/group/amibroker/files/groupfaq.html > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service. > > > ---> Outgoing mail is certified Virus Free.> Checked by AVG anti-virus system (http://www.grisoft.com).> Version: 6.0.543 / Virus Database: 337 - Release Date: 11/21/2003Send BUG REPORTS to bugs@xxxxxxxxxxxxxSend SUGGESTIONS to suggest@xxxxxxxxxxxxx-----------------------------------------Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx (Web page: http://groups.yahoo.com/group/amiquote/messages/)--------------------------------------------Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
Do you Yahoo!?
New Yahoo! Photos - easier uploading and sharing
Yahoo! Groups Sponsor
ADVERTISEMENT
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
|