[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [amibroker] Possible Problem with Walk-Forward and Custom Metrics



PureBytes Links

Trading Reference Links

Hello,

 

Thanks TJ for the quick replies.  What about AB's standard metrics? Ulcer Performance Index, Kratio. etc.  I am still having problems with the BEST not being selected to Walk Forward.  Below is an example from the data.  During the WF optimization UPI was selected a the Optimization Target.  The value it selected was 670.24 which is not the best.  It should've selected 2111.47.

 

This is just one example. It does NOT always happen.  So a few random spot checks may not reveal the problem.

 

Ulcer Index Ulcer Perf. Index Sharpe Ratio
1.33 2111.47 5.46
2.79 1329.43 5.5
1.25 1270.89 4.92
0.96 1037.77 5.22
1.11 670.24 5.29
1.51 531.86 5.33
2.56 434.98 5.27
2.18 430.98 4.99
1.07 416.38 4.2

 

Mode Begin End No. Ulcer Index Ulcer Perf. Index Sharpe Ratio
IS 4/26/2008 5/26/2008 5 2.3 865.5 4.9
OOS 5/26/2008 6/2/2008 1 1.96 14589.06 11.77
IS 5/3/2008 6/2/2008 5 2.55 739.23 5.76
OOS 6/2/2008 6/9/2008 1 0 N/A 9.63
IS 5/10/2008 6/9/2008 8 1.11 670.24 5.29

 

Regards,

 

Wade Bullock


--- On Sat, 6/14/08, Tomasz Janeczko <groups@xxxxxxxxxxxxx> wrote:

From: Tomasz Janeczko <groups@xxxxxxxxxxxxx>
Subject: Re: [amibroker] Possible Problem with Walk-Forward and Custom Metrics
To: amibroker@xxxxxxxxxxxxxxx
Date: Saturday, June 14, 2008, 6:37 AM

Hello,

Things you should check
1. Your custom metric name MUST be unique
2. The "optimization target" selected in the AA->Settings- >Walk Forward
must be typed EXACTLY as it is defined in the code, including case senstivity.
If you define "Profit" in the code, but "profit" as optimization target,
it won't recognize it.
3. Go to Tools->Preferences, "Miscellaneous" tab,
make sure that "Thousand separator" field is empty,
press OK then re-run WF.

Best regards,
Tomasz Janeczko
amibroker.com
----- Original Message -----
From: "Joe" <j0etr4der@xxxxxx com>
To: <amibroker@xxxxxxxxx ps.com>
Sent: Saturday, June 14, 2008 2:25 AM
Subject: [amibroker] Possible Problem with Walk-Forward and Custom Metrics

> If you are using a Custom Metric as your Optimization target in the
> Walk-Forward settings, you should hand-check your results. I am
> frequently getting incorrect results, in that the highest value of the
> Custom Metric is not the one selected by W-F for a date range.
>
> I documented the problem in the 5.08beta to AB support on May 13th and
> have not received a reply as yet.
>
> This is the metric that is currently causing(?) the problem:
> [code snippet]
> ProfitMetric = ( NetProfitpc + RecFactor * 20 ) * 10 ;//make them
> ~equivalant
> if( 0 > NetProfitpc AND 0 > RecFactor AND 0 < ProfitMetric )//if
> both negative & result positive
> ProfitMetric = abs( ProfitMetric );
>
> bo.AddCustomMetric( "Profit", ProfitMetric );
> [end code snippet]
>
> Other of my metrics exhibit the same type of incorrect results in my
> testing. I'm using Release 5.10.1 now, so I thought it was appropriate
> to bring this to the attention of the group.
>
> Good trading,
>
> Joe
>
>
> ------------ --------- --------- ------
>
> Please note that this group is for discussion between users only.
>
> To get support from AmiBroker please send an e-mail directly to
> SUPPORT {at} amibroker.com
>
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroke r.com/devlog/
>
> For other support material please check also:
> http://www.amibroke r.com/support. html
> Yahoo! Groups Links
>
>
>

__._,_.___

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html




Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___