|
PureBytes Links
Trading Reference Links
|
Awesome Bruce. Look forward to it!
You certainly have my curiosity piqued about the solution.
--- In amibroker@xxxxxxxxxxxxxxx, "bruce1r" <brucer@xxx> wrote:
>
> Ozzy, Howard -
>
> I thought about posting this when I saw the original note. Not
> wanting to get into the question of methodology that you are about to
> discuss, I mention that it is possible to distinguish IS from OOS. It
> is a bit of a hack, though, and I was reluctant to post it because of
> that. It works for me and I think it will work in every case that I
> can imagine, but you never know ! Anyway, I'm about to leave until
> late afternoon, but will post it when I return and can write up some
> brief usage notes.
>
> Tomasz looks like he provided a IS / OOS flag in the optimizer DLL
> interface, but for what I was interested in, I needed it in AFL and
> didn't want to mod each optimizer DLL source
>
> BTW, for me, the reason that this was so important was that it is
> critical to the use of walk-forward test results of market timing
> signals. AB splices together an OOS equity curve for you, but in
> timing signal applications, one really wants the spliced buy/sell
> signal segments also. This is easily done in the CBT - if you know
> when it is OOS.
>
> Later ...
>
> -- Bruce R
>
>
>
> --- In amibroker@xxxxxxxxxxxxxxx, "Howard B" <howardbandy@> wrote:
> >
> > Hi OAM --
> >
> > If I understand correctly, you do not need the walk forward process
> at all.
> >
> >
> > You want to set a time period and run many tests, set a new time
> period, run
> > that same group of many tests, and so forth, collecting the
results from
> > each run from each period. Yes?
> >
> > If so, you might try BatMan. It used to be in the Yahoo Group
> > "AmiBroker-ts/files", but I do not see it there today. Fred Tonetti
> is the
> > developer of BatMan. Perhaps someone can point us to the latest
> version of
> > BatMan?
> >
> > Thanks,
> > Howard
> >
> >
> >
> >
> > On Wed, Jan 14, 2009 at 4:36 PM, ozzyapeman <zoopfree@> wrote:
> >
> > > Howard, thanks for the input. I do have both of your books, btw.
> And I
> > > am aware of what you state. In 99.9% of cases what you outline is
> > > exactly what should be done.
> > >
> > > In my particular system, however, I am not searching for the "best"
> > > value, but rather a range of values that are then looped through in
> > > OOS backtesting to Buy on the best of these conditions that
happen to
> > > be "true" at the given bar. These conditions are discontinuous, and
> > > are either completely true or completely false - for e.g. GapUp(
) vs
> > > GapDown( ). Assume that I have, say, three hundred of such different
> > > "states" that I am testing buy/sell conditions against. Each
state can
> > > be assigned a variable, and I can optimize to check historical
> > > performance of each state:
> > >
> > > optimize ("a", 1, 1, 300, 1);
> > >
> > > This type of optimization must be exhaustive, as opposed to
using CMAE
> > > non exhaustive, as each state has nothing to do with it's "neighbor"
> > > state. In actuality they are more complex states than simply
GapUp or
> > > GapDown. Each state was arrived at by calculations from another
> > > program, and the order that they are tested in the optimization is
> > > somewhat random. Each one is truly unique.
> > >
> > > My AFL does the above optimization, and as it does so, I fput
the top
> > > 50 or so values of "a", according to some custom metric, to an
> > > external file. When it comes time to backtest on OOS data, I
then want
> > > to pull those top 50 values and then test to see which "states" are
> > > true (more than 1 can be true), and then Buy on the state that
has the
> > > best corresponding metric.
> > >
> > > Naturally I can do all of the above manually - e.g. run an
> > > optimization on an IS period using the 'PUSH AFL', then run a
backtest
> > > on an OOS period using the 'PULL AFL', and repeat.... But I would
> > > prefer to use the Walk-Forward engine to do all this automatically
> > > and, preferably, with a single PUSH/PULL AFL.
> > >
> > > Hope that paints a clearer picture.
> > >
> > >
> > > --- In amibroker@xxxxxxxxxxxxxxx <amibroker%40yahoogroups.com>,
> "Howard B"
> > > <howardbandy@> wrote:
> > > >
> > > > Hi OAM --
> > > >
> > > > The purpose of walk forward testing is to see what happens in the
> > > > out-of-sample period after a trading has been chosen based on
> > > performance
> > > > over an in-sample period. In walk forward processing, you (the
> system
> > > > designer) do not have an opportunity to evaluate any of the
> alternative
> > > > systems that are associated with parameter sets other than the one
> > > that is
> > > > at the top of the ranking according to your objective
function. The
> > > purpose
> > > > of the objective function is to incorporate everything that is
> > > important and
> > > > arrive at a single-valued score.
> > > >
> > > > So, the question is -- what happens during selection of the best
> of the
> > > > alternatives based on the in-sample period that tells you to
change
> > > > something (use a different system) in the out-of-sample period? If
> > > you have
> > > > a quantifiable answer to that question, incorporate whatever
> > > decision making
> > > > you would do into the original objective function and run the
tests
> > > in the
> > > > normal manner.
> > > >
> > > > Or am I missing something?
> > > >
> > > > Thanks,
> > > > Howard
> > > > www.blueowlpress.com
> > > >
> > > >
> > > >
> > > > On Wed, Jan 14, 2009 at 1:57 PM, ozzyapeman <zoopfree@> wrote:
> > > >
> > > > > Hello, I am trying to branch my code whenever the Walk-Forward
> > > engine
> > > > > is processing OOS data. Hoping someone can give me a tip on
this.
> > > > >
> > > > > Normally, when one does a Walk-Forward test, the trading
> system being
> > > > > tested should be identical for both in-sample (IS) and
> out-of-sample
> > > > > (OOS) data. But there are some rare instances when one might
> want to
> > > > > use a slightly modified version of the main trading system for
> the OOS
> > > > > data. Without going into details on my trading system, assume
> for the
> > > > > moment that the reason for doing such a thing is valid. On
> that basis,
> > > > > what I essentially want to achieve in my AFL is something like:
> > > > >
> > > > > if ( Walk-Forward engine is processing IS data)
> > > > >
> > > > > { Use Trading System A }
> > > > >
> > > > > if ( Walk-Forward engine is processing OOS data)
> > > > >
> > > > > { Use Trading System B }
> > > > >
> > > > > However, using traces with ActionEx status, indicates that the
> engine
> > > > > state for both of the above cases is always
> > > "actionExOptimizeBacktest".
> > > > >
> > > > > As such, there is no way (that I can think of) to effectuate
> the above
> > > > > branching code.
> > > > >
> > > > > Is there a way around this? Some alternate way of testing
when OOS
> > > > > data is being processed vs IS data? Or is the only practical
> solution
> > > > > to build a custom version of the Walk-Forward process using
> OLE with
> > > > > the AA objects?
> > > > >
> > > > > Thank you for any input.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> >
>
------------------------------------
**** IMPORTANT ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
*********************
TO GET TECHNICAL SUPPORT from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
*********************
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
*********************************
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|