[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: Custom Metric for Walk Forward Optimization



PureBytes Links

Trading Reference Links

Hi,

One of the goals of walk forward optimization is to remove the 
discretionary decisions (i.e. the Params) and see how your system 
would perform on out of sample (OOS) data after having been optimized 
over in sample (IS) data.

At each OOS phase, the sytem uses only one set of values; the optimal 
values of the previous IS optimization. Think of it as optimization 
followed by backtest followed by optimization followed by backtest...

As such, you probably want to convert your Param statements to 
Optimize statements, then set your custom metric as the Optimization 
Target in the Walk Forward settings tab ( 
http://www.amibroker.com/kb/category/afl/systems/ ) and select walk 
forward from the Optimize button of the AA window.

Note, your customization metric will be calculated for each 
optimization combination and used as the measure by which each 
optimization is judged when determining which single set of values to 
apply to the next OOS backtest.

Specifically; At each iteration, your script will be optimized over 
all combinations against the IS period. Then the optimal values, 
based on the best rank as per your optimization target, will be 
applied to the OOS period to give the OOS results.

Then the dates advance forward such that the OOS data becomes part of 
the IS data and the process is repeated, giving OOS results with 
possibly different optimized values at each stage.

However, the following line of your code makes no sense to me, and 
should probably just be removed, since you've already calculated your 
new value and all you need to do is set it as a custom metric:

> NewKRatio = st.getvalue("NewKRatio");

Mike

--- In amibroker@xxxxxxxxxxxxxxx, "David Fitch" <davidfitch@xxx> 
wrote:
>
> I am trying to make a custom metric for walk forward optimization 
testing. This custom metric includes parameters that need input from 
the walk forward periods. For example, if I were making a custom 
metric K Ratio, I would need the number of bars in the IS and OOS 
periods to figure Standard Error, Standard Deviation, and Linear 
Regression slope. How do I do this? And is there a better way?
> 
> Here's a custom metric AFL with parameter boxes that would be 
replaced with whatever is needed to make this work when running walk 
forward optimization.
>  
> LRPeriods=Param("LinRegPeriods",10,1,500,1);
> LR=LinRegSlope(e,LRPeriods);
> 
> SEPeriods=Param("SEPeriods",10,1,500,1);
> 
> SE=StdErr(e,SEPeriods);
> 
> SDPeriods=Param("SDPeriods",10,1,500,1);
> 
> SD=StDev(e,SDPeriods);
> 
> NewKRatio= LR / ((SE/SD)* sqrt(SDPeriods));
> 
> SetCustomBacktestProc("");
> 
> if (Status("Action") == actionPortfolio)
> 
> {
> 
> bo = GetBacktesterObject();
> 
> bo.backtest();
> 
> st = bo.getperformancestats(0);
> 
> NewKRatio = st.getvalue("NewKRatio");
> 
> bo.addcustommetric("NewKRatio", NewKRatio);
> 
> }
> 
> Thanks
> 
> Dave
>



------------------------------------

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/