|
PureBytes Links
Trading Reference Links
|
If it is your CAR for OOS then your system is probably good. Look at your other KPIs such as CAR/Maxdd for OOS, that should be above 2. You have to calculate it manually as it is unfortunately not available directly from the WF report. Also your Walk Forward Efficiency, e.g. CAR_OOS/CAR_IS should be greater than 50%.
I use IS:OOS period 4:1. My trades in IS are at least 50.
--- In amibroker@xxxxxxxxxxxxxxx, "Gonzaga" <gonzagags@xxx> wrote:
>
> Thanks, Howard for your knowledge: I've read your book, I think is very good.
> But after all, one have to deal with his own systems and problems.
> In my case, after these discussions, I have increased the IS period from 3 months to 5 months.
> The CAR in the WF simulation have increased from 30% to 45%!!!!
>
> the OS period is still in 3 months. I will reduce to 1 month, which will probably increase the CAR, but the computer takes soooo muuuuch time to perform such a simulation..
> greetings
>
> --- In amibroker@xxxxxxxxxxxxxxx, Howard B <howardbandy@> wrote:
> >
> > Greetings all --
> >
> > My point of view on the length of the in-sample and out-of-sample may be a
> > little different.
> >
> > The logic of the code has been designed to recognize some pattern or
> > characteristic of the data. The length of the in-sample period is however
> > long it takes to keep the model (the logic) in synchronization with the
> > data. There is no one answer to what that length is. When the pattern
> > changes, the model fits it less well. When the pattern changes
> > significantly, the model must be re-synchronized. The only person who can
> > say whether the length is correct or should be longer or shorter is the
> > person running the tests.
> >
> > The length of the out-of-sample period is however long the model and the
> > data remain in sync. That must be some length of time beyond the in-sample
> > period in order to make profitable trades. It could be a long time, in
> > which case there is no need to modify the model at all during that period.
> > There is no general relationship between the length of the in-sample period
> > and the length of the out-of-sample period -- none. There is no general
> > relationship between the performance in-sample and the performance
> > out-of-sample. The greater the difference between the two, the better the
> > system has been fit to the data over the in-sample period. But that does
> > not necessarily mean that the out-of-sample results are less meaningful.
> >
> > You can perform some experiments to see what the best in-sample length is.
> > And then to see what the typical out-of-sample length is. Knowing these
> > two, set up a walk forward run using those lengths. After the run is over,
> > ignore the in-sample results. They have no value in estimating the future
> > performance of the system. It is the out-of-sample results that can give
> > you some idea of how the system might act when traded with real money.
> >
> > It is nice to have a lot of closed traded in the out-of-sample period, but
> > you can run statistics on as few as 5 or 6. Having fewer trades means that
> > it will be more difficult to achieve statistical significance. The number
> > 30 is not magic -- it is just conventional.
> >
> > I think it helps to distinguish between the in-sample and out-of-sample
> > periods this way -- in-sample is seeing how well the model can be made to
> > fit the older data, out-of-sample is seeing how well it might fit future
> > data.
> >
> > Ignore the television ads where person after person exclaims "backtesting!"
> > as though that is the key to system development. It is not. Backtesting by
> > itself, without going on to walk forward testing, will give the trading
> > system developer the impression that the system is good. In-sample results
> > are always good. We do not stop fooling with the system until they are
> > good. But in-sample results have no value in predicting future performance
> > -- none.
> >
> > There are some general characteristics of trading systems that make them
> > easier to validate. Those begin with having a positive expectancy -- no
> > system can be profitable in the long term unless it has a positive
> > expectancy. Then going on to include trade frequently, hold a short time,
> > minimize losses. Of course, there have been profitable systems that trade
> > infrequently, hold a long time, and suffer deep drawdowns. It is much
> > harder to show that those were profitable because they were good rather than
> > lucky.
> >
> > There is more information about in-sample, out-of-sample, walk forward
> > testing, statistical validation, objective functions, and so forth in my
> > book, "Quantitative Trading Systems."
> > http://www.quantitativetradingsystems.com/
> >
> > Thanks for listening,
> > Howard
> >
> > On Sun, Oct 4, 2009 at 10:56 AM, Bisto <bistoman73@> wrote:
> >
> > >
> > >
> > > Yes, I believe that you should increase the IS period
> > >
> > > as general rule is not true "the shortest the best" trying to catch every
> > > market change because it's possible that a too short IS period produces a
> > > too low number of trades with no statistical robustness --> you will find
> > > parameters that are more likely candidated to fail in OS
> > >
> > > try a longer IS period and let's see what will happen
> > >
> > > I read an interesting book on this issue: "The evaluation and optimization
> > > of trading strategies" by Pardo. Maybe he repeated too much times the same
> > > concepts nevertheless I liked it
> > >
> > > if anyone could suggest a better book about this issue it would be very
> > > appreciated
> > >
> > >
> > > Bisto
> > >
> > > --- In amibroker@xxxxxxxxxxxxxxx <amibroker%40yahoogroups.com>, "Gonzaga"
> > > <gonzagags@> wrote:
> > > >
> > > > Oh, sorry, I am lost in translation ... ;-)
> > > > Yes I meant trades of my IS period.
> > > > I've got about 70 trades in my IS period, three months.
> > > > BUT, I buy stocks in a multiposition way.This means, that my hole capital
> > > divides among several stocks purchased simultaneously.
> > > > So, in my statistics, I use to average my trades. When I use
> > > maxopenpositions=7, I use to average my results every 7 trades.
> > > > Considering that, my trades in three months are not 70, but less ( not
> > > exactly 70/7, but less than 70)
> > > >
> > > > If I use maxopenposition=1, which is, invest all my capital every trade,
> > > in three months I would have about 29 trades.
> > > > So I suppose I have to increase the IS period.. isn`t it?
> > > >
> > > >
> > > > --- In amibroker@xxxxxxxxxxxxxxx <amibroker%40yahoogroups.com>, "Bisto"
> > > <bistoman73@> wrote:
> > > > >
> > > > > What do you mean with "I don't have many buyings and sellings"?
> > > > >
> > > > > If you have less than 30 trades in an IS period, IMHO, you are using a
> > > too short period due to not statistical robustness --> WFA is misleading,
> > > try a longer IS period
> > > > >
> > > > > Bisto
> > > > >
> > > > > --- In amibroker@xxxxxxxxxxxxxxx <amibroker%40yahoogroups.com>,
> > > "Gonzaga" <gonzagags@> wrote:
> > > > > >
> > > > > > Thanks for the answers
> > > > > > To Keith McCombs :
> > > > > >
> > > > > > I use 3 months IS test and 1 month step, this is, 1 month OS test. My
> > > system is an end-of day-system, so I don't have many buyings and sellings..
> > > > > > Perhaps I should make bigger the IS period?
> > > > > >
> > > > > > anyway, my parameter behaves well in any period. Of course it is an
> > > optimized variable, but it doesn't fail in ten years, in none of those ten
> > > years, over 500 stocks.. a very long period..
> > > > > > So, couldn't it be better, on the long run, than the parameters
> > > optimized with the WF study?
> > > > > > (In fact, I am using it now, the optimized variable)
> > > > > > That's my real question..
> > > > > >
> > > > > > To dloyer123:
> > > > > > I haven't understood the meaning of the Walk Forward Efficency, and
> > > seems interesting.
> > > > > > can you explain it better, please..?
> > > > > >
> > > > > >
> > > > > >
> > > > > > --- In amibroker@xxxxxxxxxxxxxxx <amibroker%40yahoogroups.com>,
> > > "dloyer123" <dloyer123@> wrote:
> > > > > > >
> > > > > > > I have had similar experiences. I like to use WFT to estimate what
> > > Pardo call's his "Walk Forward Efficency", or the ratio of the out of sample
> > > WF profits to just optimizing over the entire time period.
> > > > > > >
> > > > > > > A good system should have as high a WFE as posible. Systems with a
> > > poor WFE tend to do poorly in live trading.
> > > > > > >
> > > > > > > If you have a parm set that works well over a long period of live
> > > trading, then you are doing well!
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> >
>
------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|