|
PureBytes Links
Trading Reference Links
|
Thanks Mike, but I was wondering if anyone had explored weighting the
in sample period of a walkforward test and if they found it to be
useful.
My current custom fitness function invokes a dll that performs the
bulk of the backtest on the gpu. Very fast, but a PITA to modify.
--- In amibroker@xxxxxxxxxxxxxxx, "Mike" <sfclimbers@xxx> wrote:
>
> AmiBroker supports the use of any backtest metric as the fitness
> function, including custom metrics. As such, you can write your own
> custom metric and use it as the fitness function.
>
> To add a custom metric:
> http://www.amibroker.com/guide/a_custommetrics.html
>
> To make that metric the fitness function:
> http://www.amibroker.com/kb/2008/02/12/getting-started-with-
automatic-
> walk-forward-optimization/
>
> Notice the quote "You can also TYPE-IN any custom metric that you
> have added via custom backtester interface".
>
> Mike
>
> --- In amibroker@xxxxxxxxxxxxxxx, "dloyer123" <dloyer123@> wrote:
> >
> > Very good point.
> >
> > Is there a good way to gradually decrease the impact of older
data
> on
> > the system during walkforward data?
> >
> > The data expiring off the back end of the walkforward in sample
> > period has just as much impact on the optimal values as the new
> data
> > collected today.
> >
> > Wouldn't it be better give more importance and less importance to
> > older data. Would that not reduce the system "lag" in response
to
> > changing market conditions?
> >
> > For example, last aug was a very "interesting" time. With a 1
year
> > in sample period, that data would be aging off now and could
result
> > in a step change to the optimal system values.
> >
> > If we could somehow use a fitness function that devalued older
> traded
> > compared to newer trades, it might make the system adapt more
> quickly
> > and avoid rapid changes due to old data aging out of the in
sample
> > period.
> >
> >
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "Howard B" <howardbandy@> wrote:
> > >
> > > Greetings all --
> > >
> > > If the walk forward tests show poor out-of-sample results for
the
> > entire
> > > out-of-sample period, that means the model is learning noise,
not
> > signal.
> > > If they are good for a short period out-of-sample, then fall
off,
> > that means
> > > the characteristics of the market being modeled are changing
and
> > you need
> > > more frequent reoptimization -- a shorter out-of-sample time
> period.
> > >
> > > If you have a model that has in the past been profitable out-of-
> > sample, and
> > > it becomes unprofitable during a period of real trading, you
can
> > reoptimize
> > > ahead of schedule. In fact, if you have valid system, you can
> > reoptimize as
> > > frequently as you want to. You might want to run some walk
> forward
> > tests to
> > > verify that this works for your system. If you have not tested
> > early
> > > reoptimizing, but do it to try to resync a system that is in a
> > drawdown, I
> > > recommend paper trading the new parameter set for a while to be
> > certain that
> > > the model just needed re-synchronizing, and was not terminally
> > broken.
> > >
> > > Thanks,
> > > Howard
> > >
> > >
> > >
> > > On Mon, Sep 8, 2008 at 2:05 PM, Mike <sfclimbers@> wrote:
> > >
> > > > Sometimes it is not so much adaptation that is needed. But
> > rather just
> > > > capturing the transition of cycles.
> > > >
> > > > Consider a house located in a region that has warm summers
and
> > cold
> > > > winters. The variables would be climate control (e.g. air
> > > > conditioner/heater). The variable values will change as the
> > seasons
> > > > unfold. To optimize comfort, we will be using increased air
> > > > conditioning during the summers and increased heating during
the
> > > > winters.
> > > >
> > > > By experimenting with OOS time periods we may be able to
> identify
> > > > market cycles. If you only reoptimized your values once a
year
> > (e.g.
> > > > set for average temperature for the year), your climate
control
> > would
> > > > likely be generally too warm in the summers and generally too
> > cold in
> > > > the winters. But, if you reoptimized monthly, then you would
> > gradually
> > > > use increasing air conditioning leading into the summer and
> > conversely
> > > > gradually increase heating leading into the winter.
> > > >
> > > > Obviously the markets are not so perfectly cyclical as the
> > seasons.
> > > > But, the example illustrates the point.
> > > >
> > > > Mike
> > > >
> > > >
> > > > --- In amibroker@xxxxxxxxxxxxxxx <amibroker%
> > 40yahoogroups.com>, "Ara
> > > > Kaloustian" <ara1@> wrote:
> > > > >
> > > > > WF testing seems like a logical step to take in creating a
> > trading
> > > > system. While I am not suggesting that we don't do it, I'd
like
> to
> > > > clear my understanding.
> > > > >
> > > > > It seems to me that we are looking to create a system with
> > > > parameters that are valid "for all time". If a test is
> succesful
> > in
> > > > OOS period, then we can assume that we have found something
that
> > > > remains applicable for the future - at least in this one
> instance.
> > > > >
> > > > > Given that the markets change all the time, the obvious
> > conclusion
> > > > for me is that we need indicators that are adaptive... and
that
> > any
> > > > that do not adapt will simply not work in OOS.
> > > > >
> > > > > The issue of providing feedback from equity curve seems
valid
> > and it
> > > > can provide a warning about system starting to not perform as
> > expected
> > > > ... so regardless of type of indicators used, this kind of
> > feedback is
> > > > good!.
> > > > >
> > > > > The point I am making is that there has been very little
said
> > about
> > > > adaptive indicators ...
> > > > >
> > > > > maybe that is the holy grail ... and therefore
untennable ...
> > > > >
> > > >
> > > >
> > > >
> > >
> >
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|