|
PureBytes Links
Trading Reference Links
|
Hi Keith,
And many thanks for your excellent commonsense suggestions. Following this up has proved to be a useful learning experience (including a fair amount of re-learning!).
The first lesson is that one should always do some optimisation and backtesting before leaping into a full-blown WF analysis. The second point is that I should not expect max profit when optimising on a different parameter - RAR/MDD in this case. However, I would not have expected the profit to be too far down from the max - though this may be wishful thinking!.
In case anyone is interested this is a summary of what I found by optimising individual steps from the WF process:-
The detailed results and the 3-D graph for my IS1 period showed large plateau areas, with over 100 cases giving the same (max) RAR/MDD. Most of these had the same profit figure (533), but another group (still with the same RAR/MDD) had a lower profit figure. Lesson no.3 was that my optimisation steps were closer spaced than necessary.
Similarly, in IS2 a large number of cases had the same RAR/MDD (16.52) and profit(474). Another group had RAR/MDD at 14.75 and P of 360. Then a third set, and then a fourth with RAR/MDD at 13.14 and P at 1126. This was the profit figure that showed up in the OOS1 results in the WF. It remains a puzzle to me, because a Backtest in OOS1 using the optimum parameters from IS1 gave a profit of 474. It might remain an unsolved mystery.
Finally, I guess that with a system like this that produces only a small number of trades in a 6 month period, I should consider a more complex fitness function - perhaps (e.g.) combining RAR/MDD with Profit in some way.
Thanks again
Brian
--- In amibroker@xxxxxxxxxxxxxxx, Keith McCombs <kmccombs@xxx> wrote:
>
> Brain --
> I don't know what the problem or the solution is. However, this should
> be fairly easy for you to debug.
>
> All you should have to do is repeat the walk forward process, one step
> at a time:
> 1. Optimize for IS1. Then using the resulting parameters, backtest
> IS1, and record the trades.
> 2. Using same parameters, backtest OOS1, and record the trades.
> 3. Compare with your WF results. You should have enough information
> now to find the problem.
> 4. If after 3 above, you still can't figure it out, then optimize OOS1
> and IS2, which should produce identical results. If not, why not?
>
> The secret to debugging is "divide and conquer", just "one step at a time".
> -- Keith
>
> brianw468 wrote:
> >
> >
> > Oops,
> > This message became a mess as Yahoo apparently removed most of the
> > spaces, destroying the layout..
> > Results in the format (Sample, Profit, No. of Trades) are:
> > (IS1,533,1) (OOS1,1126,2) (IS2,474,1) (OOS2,-537,2) (IS3,43,1)
> > (OOS3,784,2).
> > I hope this comes out a bit clearer!
> >
> > Brian
> >
> > --- In amibroker@xxxxxxxxxxxxxxx <mailto:amibroker%40yahoogroups.com>,
> > "brianw468" <wild21@> wrote:
> > >
> > > Hi
> > > I recently tested a trading system but got some strange results from
> > the WF test. The WF process used exhaustive optimization.
> > > An extract from the results (too large to reproduce in full) is:-
> > > Item Profit No. Trades
> > > IS1 533 1
> > > OOS1 1126 2
> > >
> > > IS2 474 1
> > > OOS2 -537 2
> > >
> > > IS3 43 1
> > > OOS 784 2
> > >
> > > In two cases the OOS results are better than the preceding IS
> > results. Though a little surprising,this is quite plausible, because
> > the time periods differ.
> > > However,the IS2 profit is much poorer than the OOS1 profit, and this
> > is a worry since exhaustive optimization is used, and the time periods
> > are the same. Can anyone offer an explanation? Would it lie in the way
> > profit is handled when a trade is not completed by the end of the
> > selected period?
> > >
> > > Thanks
> > > Brian
> > >
> >
> >
>
------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|