|
Hi Joe,
Funny you should write this today. I too have been having
similar problems with the WF. I was about to write a message
when I came across yours.
Back in March I reported the problem to support as I was
having trouble with not only custom metrics that I was trying to
develop but also AB standard metrics. To date, I've have had
trouble using as an Optimization Target: Net Profit, Kratio, and Ulcer
Performance Index.
My problem is that the WF does not always take the "best"
Optimization Target. Often it will but not always.
I originally discovered the problem while trying to develop some
Custom Metrics. I even posted a message (#120147) Even though TJ
approved the code the problem was the same. Below is that
message.
"I've added a simple custom metric via the custom backtester
interface
and while the custome metric shows up as a column in the
optimization
report, it does not sort the results and find
the best one--even
though I have TYPED it into the
"Optimization Target" field.
Wondering if I'm missing something
from my simple code
below.
SetCustomBacktestProc("");
if( Status("action")
== actionPortfolio )
{
bo =
GetBacktesterObject();
bo.Backtest();
st =
bo.GetPerformanceStats(0);
npkr = st.GetValue("NetProfit") *
st.GetValue("Kratio");
bo.AddCustomMetric( "NPKR", npkr
);
}"
I followed TJ's recommendations from his reply but that didn't
help.
After that I discovered there was not only a problem with CM's but
also Standard Metrics. I documented the problem to support but
they said they could not reproduce the problem.
[#47484] Walk Forward and Optimization Target
So I dropped it for a couple of months while I worked on some other
things, but am back wondering what the problem is. I've spent
the last couple of days looking for problems on my end but cant'
find any or don't see them.
Now, I also have release 5.10.1
Any help would be greatly appreciated.
Wade Bullock
--- On Fri, 6/13/08, Joe
<j0etr4der@xxxxxxxxx> wrote:
From:
Joe <j0etr4der@xxxxxxxxx>
Subject: [amibroker] Possible
Problem with Walk-Forward and Custom Metrics
To:
amibroker@xxxxxxxxxxxxxxx
Date: Friday, June 13, 2008, 5:25
PM
If you are using a Custom Metric as your Optimization target in
the
Walk-Forward settings, you should hand-check your results. I
am
frequently getting incorrect results, in that the highest
value of the
Custom Metric is not the one selected by W-F for a
date range.
I documented the problem in the 5.08beta to AB
support on May 13th and
have not received a reply as
yet.
This is the metric that is currently causing(?) the
problem:
[code snippet]
ProfitMetric = ( NetProfitpc +
RecFactor * 20 ) * 10 ;//make them
~equivalant
if( 0 >
NetProfitpc AND 0 > RecFactor AND 0 < ProfitMetric
)//if
both negative & result positive
ProfitMetric = abs(
ProfitMetric );
bo.AddCustomMetric( "Profit", ProfitMetric
);
[end code snippet]
Other of my metrics exhibit the same
type of incorrect results in my
testing. I'm using Release 5.10.1
now, so I thought it was appropriate
to bring this to the
attention of the group.
Good
trading,
Joe
|