|
PureBytes Links
Trading Reference Links
|
Fred- Don't quote me out of context. That was an EXAMPLE. You left
out the part before it: "Let's say... you choose to break the data
into 10 segments and do the initial optimization on the first three
segments." Also that there would be SEVEN iterative optimizations and
OOS testings from there. I really don't think you can help. Thanks
anyway.
Mark
--- In amibroker@xxxxxxxxxxxxxxx, "Fred" <fctonetti@xxxx> wrote:
> Mark,
>
> Fine ... I can ignore all of that except ...
>
> "The code would divide the data into 10 (near)equal segments, then do
> the initial optimization on the first 3. It would then take the best
> performing period (I'd like this in terms of profit factor) and test
> it on segment #4 (which is the first OOS segment)" ... if that
> doesn't imply picking which of the n segments is best or the
> parameter values used therein then can you tell me what that does
> mean ? By period do you mean value for the parameter ?
>
> --- In amibroker@xxxxxxxxxxxxxxx, "MarkF2" <feierstein@xxxx> wrote:
> > Fred, Please read my post again more carefully. I've said nothing
> > about Active Trader, 25 bars, or picking which of the n segments is
> > best. If the description I've provided isn't clear enough, then I
> > don't know what else to say.
> >
> > Mark
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "Fred" <fctonetti@xxxx> wrote:
> > > Mark,
> > >
> > > First a statement ... There may !? be some ?! validity to walk
> > > forward testing using either an anchored original data point or a
> > > moving window but clearly not with windows of 25 bars. Your
> > > description of how you'd like to do this is the first I've ever
> seen
> > > of this type i.e. pick which of the n-segments is best and use
> that.
> > > This could be done but I would think ?! it would be best to
> either
> > > use the entire period from the anchored first point to the
> current
> > > bar or a sliding window of n bars from n-1 bars ago until the
> current
> > > bar to base the optimization on. If your looking for a
> methodology
> > > that actually might have some promise to it I suspect one of
> these
> > > methods would provide better results. If you are just looking to
> > > dispell what was in Active Trader then I think that 1. They've
> > > pretty much already done this themselves by virtue of the size of
> > > their window and 2. this MIGHT be a waste of time in that no one
> here
> > > will really benefit from the outcome except that one is sure not
> to
> > > use AT's methods.
> > >
> > > Your thoughts ?
> > >
> > > Fred
> > >
> > > --- In amibroker@xxxxxxxxxxxxxxx, "MarkF2" <feierstein@xxxx>
> wrote:
> > > > Fred, OK, here's a more detailed description.
> > > >
> > > > The code as I envision it would allow the walk-forward OOS
> testing
> > > of
> > > > a system with one optimization parameter in an exploration. For
> > > > example:
> > > >
> > > > per = optimize("period",10,2,20,2);
> > > > Buy = Cross(C,EMA(C,per));
> > > > Sell = Cross(EMA(C,per),C);
> > > > Short = Sell;
> > > > Cover = Buy;
> > > >
> > > > The loop code would have an area to paste the system and would
> allow
> > > > two additional inputs: 1) # of segments to break the data into
> and
> > > 2)
> > > > # of segments to use for the initial optimization. If the
> system
> > > must
> > > > be optimized *by the loop*, then it would also need a mechanism
> for
> > > > inputting the optimization parameters. It may be easier to
> write a
> > > > separate routine for sliding optimization windows, I don't
> know.
> > > I'm
> > > > just describing what I want. But I can tell you that in most
> cases,
> > > > sliding windows are just too noisy to give a meaningful
> assessment
> > > of
> > > > *overall system robustness* (which is what I'm testing for)
> because
> > > > you will end up optimizing on a bearish segment and then OOS
> testing
> > > > on a bullish or ranging segment, etc.
> > > >
> > > > Let's say the system doesn't have to be optimized by the loop
> and
> > > you
> > > > choose to break the data into 10 segments and do the initial
> > > > optimization on the first three segments.
> > > >
> > > > The code would divide the data into 10 (near)equal segments,
> then do
> > > > the initial optimization on the first 3. It would then take the
> best
> > > > performing period (I'd like this in terms of profit factor) and
> test
> > > > it on segment #4 (which is the first OOS segment). Then it would
> > > > record two things for this and every other OOS segment: 1) the
> > > period
> > > > used and 2)the result (profit factor). Then it would reoptimize
> the
> > > > system over segments 1-4 and use the best period on segment 5
> (the
> > > > second OOS segment), recording the period used and result, then
> > > > reoptimize over segments 1-5, use the best period on segment 6,
> etc.
> > > >
> > > > At bare bones minimum, the exploration would have 3 columns:
> Ticker,
> > > > Period, and Profit Factor, with a row for each OOS segment.
> Columns
> > > > for the dates of each segment would be nice also, if doable.
> So if
> > > > there are 7 OOS segments and you're testing a system on 100
> stocks,
> > > > you'd get an exploration with 700 rows.
> > > >
> > > > Let me know if any questions.
> > > >
> > > > Mark
> > > >
> > > > --- In amibroker@xxxxxxxxxxxxxxx, "Fred" <fctonetti@xxxx> wrote:
> > > > > Mark,
> > > > >
> > > > > It was not my intention to be a smartass although I can see
> > > that's
> > > > > how your taking my response. If you and possibly others are
> > > > > interested in this sort of AFL, I'd be happy to look into
> writing
> > > > one
> > > > > as there may be benefits that I get out of this as well. I
> would
> > > > > certainly think that it would be possible to implement
> whether
> > > the
> > > > > period of optimization had a sliding window or an anchored
> > > > beginning
> > > > > based on a parameter. If you'd like to proceed please state
> > > > > precisely what your requirements are. After that I will
> probably
> > > > > have questions.
> > > > >
> > > > >
> > > > > --- In amibroker@xxxxxxxxxxxxxxx, "MarkF2" <feierstein@xxxx>
> > > wrote:
> > > > > > I was looking for help in implementing what I described,
> not a
> > > > > > critique (there IS a benefit in "anchoring" the front data
> > > point
> > > > if
> > > > > > you consider the results accordingly, BTW) and not a
> general
> > > > > statement
> > > > > > that it can be done "with a properly written AFL." Thanks
> > > anyway.
> > > > > >
> > > > > > --- In amibroker@xxxxxxxxxxxxxxx, "Fred" <fctonetti@xxxx>
> wrote:
> > > > > > > I can't say as I see any benefit in anchoring the front
> data
> > > > > point as
> > > > > > > this causes current data to be less and less important as
> > > time
> > > > > goes
> > > > > > > along. In addition I don't see why with a properly
> written
> > > AFL
> > > > > there
> > > > > > > would be any need for human intervention. Basically you
> > > supply
> > > > > the
> > > > > > > system and it does the rest much like the way
> PortfolioTrader
> > > > > that I
> > > > > > > posted uses a user supplied scoring routine to determine
> > > which
> > > > > > > securities to invest this would need to decide what
> parameter
> > > > > values
> > > > > > > to use as time rolls along.
> > > > > > >
> > > > > > > --- In amibroker@xxxxxxxxxxxxxxx, "MarkF2"
> <feierstein@xxxx>
> > > > > wrote:
> > > > > > > > But how about doing it the way I described? I would
> prefer
> > > to
> > > > > > > > minimize human interaction and maximize automation.
> > > Imagine
> > > > > > > testing a
> > > > > > > > system in this manner on all SP500 stocks, for example.
> > > > > > > >
> > > > > > > > --- In amibroker@xxxxxxxxxxxxxxx, "Fred"
> <fctonetti@xxxx>
> > > > wrote:
> > > > > > > > > This would be simple enough to do if you think about
> it
> > > > terms
> > > > > of
> > > > > > > how
> > > > > > > > > many bars you are going to use for a lookback period
> and
> > > > then
> > > > > > > armed
> > > > > > > > > with the results of optimization of that period how
> far
> > > > ahead
> > > > > > > from
> > > > > > > > > that point in time you are going to trade before you
> > > > > reoptimize.
> > > > > > > As
> > > > > > > > > Dingo says this could be done with automation or it
> could
> > > > all
> > > > > be
> > > > > > > done
> > > > > > > > > interally in AB/AFL/ABTool. This is a different form
> of
> > > > > Score
> > > > > > > and
> > > > > > > > > Rank if you will where instead of scoring and ranking
> > > > > individual
> > > > > > > > > issues in a portfolio on a rolling basis you are
> scoring
> > > > and
> > > > > > > ranking
> > > > > > > > > parameters for your timing system on a rolling
> basis.
> > > > > > > > >
> > > > > > > > > --- In amibroker@xxxxxxxxxxxxxxx, "dingo" <dingo@xxxx>
> > > > wrote:
> > > > > > > > > > perhaps I can help - contact me privately at dingo
> at
> > > > > udsnet
> > > > > > > dot
> > > > > > > > > com.
> > > > > > > > > >
> > > > > > > > > > d
> > > > > > > > > >
> > > > > > > > > > -----Original Message-----
> > > > > > > > > > From: MarkF2 [mailto:feierstein@x...]
> > > > > > > > > > Sent: Tuesday, June 17, 2003 4:00 PM
> > > > > > > > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > > > > > > > Subject: [amibroker] Walk-Forward Out of Sample
> (OOS)
> > > > > Testing
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Jayson, I believe Owen wants to use AB for walk-
> > > forward,
> > > > > out of
> > > > > > > > > sample
> > > > > > > > > > testing. I'd love to be able to do this also but
> lack
> > > > the
> > > > > > > > > programming
> > > > > > > > > > skills. Can anyone help? I think this would be
> > > > incredibly
> > > > > > > useful!!!
> > > > > > > > > >
> > > > > > > > > > What I'd like to do can, I believe, be done with
> loops
> > > > and
> > > > > > > would go
> > > > > > > > > > something like this:
> > > > > > > > > >
> > > > > > > > > > Let's say you're testing a simple system on MSFT
> where
> > > > > price
> > > > > > > crosses
> > > > > > > > > > over and under a moving average so you're
> optimizing
> > > only
> > > > > one
> > > > > > > > > > parameter, length, to keep things simple. The loop
> code
> > > > > would
> > > > > > > allow
> > > > > > > > > > two inputs: 1) # of segments to break data into and
> 2) #
> > > > of
> > > > > > > segments
> > > > > > > > > > to use for initial optimization. Let's say you
> choose
> > > 10
> > > > > and
> > > > > > > 3. AB
> > > > > > > > > > would divide MSFT data into 10 equal segments, then
> do
> > > > the
> > > > > > > initial
> > > > > > > > > > optimization on the first 3. It would then take
> the
> > > best
> > > > > > > performing
> > > > > > > > > > length and test it on segment #4 (which is the
> first
> > > OOS
> > > > > > > segment).
> > > > > > > > > > Then it would record two things for this and every
> > > other
> > > > > OOS
> > > > > > > > > segment:
> > > > > > > > > > 1) the length used and 2)the result (I'd like
> profit
> > > > > factor).
> > > > > > > Then
> > > > > > > > > AB
> > > > > > > > > > would reoptimize the system over segments 1-4 and
> use
> > > the
> > > > > best
> > > > > > > > > > parameter on segment 5 (the second OOS segment),
> > > > recording
> > > > > > > length
> > > > > > > > > used
> > > > > > > > > > and result, then reoptimize over segments 1-5, use
> the
> > > > best
> > > > > > > > > parameter
> > > > > > > > > > on segment 6, etc.
> > > > > > > > > >
> > > > > > > > > > What this would do is automatically tell you how
> robust
> > > > > your
> > > > > > > system
> > > > > > > > > > concept is over the OOS segments (4-10 in this
> case).
> > > > > > > > > >
> > > > > > > > > > Mark
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --- In amibroker@xxxxxxxxxxxxxxx, "Jayson"
> > > > <jcasavant@xxxx>
> > > > > > > wrote:
> > > > > > > > > > > Owen,
> > > > > > > > > > > Am I over simplifying by suggesting you simply
> use the
> > > > > > > > > > > From-To range settings in AA?
> > > > > > > > > > >
> > > > > > > > > > > Example choose 12/1/2002-12/25/2002 to optimize
> then
> > > > > simply
> > > > > > > move
> > > > > > > > > the
> > > > > > > > > > window
> > > > > > > > > > > forward 25 days to test the results.......
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > > Jayson
> > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > From: Owen Davies [mailto:owen5819@x...]
> > > > > > > > > > > Sent: Tuesday, June 17, 2003 2:04 PM
> > > > > > > > > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > > > > > > > > Subject: [amibroker] Annoying programming problem:
> > > > ideas
> > > > > > > please?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Active Trader has taken to doing something I've
> > > always
> > > > > > > considered
> > > > > > > > > > BS. Some
> > > > > > > > > > > articles--at least one a month of late--optimize
> an
> > > > > intraday
> > > > > > > > > trading
> > > > > > > > > > system,
> > > > > > > > > > > usually some variation on a volatility breakout,
> over
> > > a
> > > > > month
> > > > > > > or
> > > > > > > > > so
> > > > > > > > > > of data
> > > > > > > > > > > and then do an out-of-sample test for the next
> month
> > > or
> > > > > so to
> > > > > > > > > > "prove" how
> > > > > > > > > > > well it works. Over time, this has come to
> irritate
> > > me
> > > > > > > enough
> > > > > > > > > that
> > > > > > > > > > I'd like
> > > > > > > > > > > to do a long-term study, either to prove that
> it's
> > > > > nonsense
> > > > > > > or to
> > > > > > > > > > learn
> > > > > > > > > > > something new and highly unlikely. So:
> > > > > > > > > > >
> > > > > > > > > > > Can anyone think of a way to optimize a technique
> on,
> > > > > say, 25
> > > > > > > bars
> > > > > > > > > > of data,
> > > > > > > > > > > test it on the next 25 bars, and then step the
> window
> > > > > forward
> > > > > > > and
> > > > > > > > > > do it
> > > > > > > > > > > again? I'd settle for re-optimizing daily on the
> > > > > previous 25
> > > > > > > (or
> > > > > > > > > > whatever)
> > > > > > > > > > > bars. Whatever is easiest.
> > > > > > > > > > >
> > > > > > > > > > > For the sake of simplicity, something that runs
> on
> > > EOD
> > > > > data
> > > > > > > will
> > > > > > > > > do.
> > > > > > > > > > I mean
> > > > > > > > > > > to test breakouts in the direction of an existing
> > > trend
> > > > > and
> > > > > > > close
> > > > > > > > > at
> > > > > > > > > > the end
> > > > > > > > > > > of the day, so there should be no problem with
> days
> > > > that
> > > > > > > break out
> > > > > > > > > > in one
> > > > > > > > > > > direction, reverse, and break out in the other; a
> day
> > > > > that
> > > > > > > > > reverses
> > > > > > > > > > and ends
> > > > > > > > > > > badly will just count as a loss. Also, I can do
> the
> > > > > > > conversion to
> > > > > > > > > > intraday
> > > > > > > > > > > data myself, rather than ask others to hand me
> the
> > > > > complete
> > > > > > > > > package.
> > > > > > > > > > All I
> > > > > > > > > > > need is some way to optimize on a window.
> > > > > > > > > > >
> > > > > > > > > > > Offhand, I can't see any way to do it within AFL,
> and
> > > I
> > > > > don't
> > > > > > > have
> > > > > > > > > > the skill
> > > > > > > > > > > to handle it with external programming.
> > > > > > > > > > >
> > > > > > > > > > > Many thanks.
> > > > > > > > > > >
> > > > > > > > > > > Owen Davies
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Yahoo! Groups Sponsor
> > > > > > > > > > > ADVERTISEMENT
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Send BUG REPORTS to bugs@xxxx
> > > > > > > > > > > Send SUGGESTIONS to suggest@xxxx
> > > > > > > > > > > -----------------------------------------
> > > > > > > > > > > Post AmiQuote-related messages ONLY to:
> > > > > > > amiquote@xxxxxxxxxxxxxxx
> > > > > > > > > > > (Web page:
> > > > > http://groups.yahoo.com/group/amiquote/messages/)
> > > > > > > > > > > --------------------------------------------
> > > > > > > > > > > Check group FAQ at:
> > > > > > > > > > >
> > > > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > > > > > > > > > >
> > > > > > > > > > > Your use of Yahoo! Groups is subject to the Yahoo!
> > > > Terms
> > > > > of
> > > > > > > > > Service.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Yahoo! Groups Sponsor
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > <http://rd.yahoo.com/M=251812.3170658.4537139.12
> > > > 61774/D=egroupweb/S=17
> > > > > > > > > 05
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > 632198:HM/A=1564416/R=0/SIG=11ti81skc/*http://ww
> > > > w.netflix.com/Default?
> > > > > > > > > mq
> > > > > > > > > > so=60164797&partid=3170658>
> > > > > > > > > >
> > > > > > > > > > <http://us.adserver.yahoo.com/l?
> > > > > > > > > M=251812.3170658.4537139.1261774/D=egrou
> > > > > > > > > > pmail/S=:HM/A=1564416/rand=707349073>
> > > > > > > > > >
> > > > > > > > > > Send BUG REPORTS to bugs@xxxx
> > > > > > > > > > Send SUGGESTIONS to suggest@xxxx
> > > > > > > > > > -----------------------------------------
> > > > > > > > > > Post AmiQuote-related messages ONLY to:
> > > > > > > amiquote@xxxxxxxxxxxxxxx
> > > > > > > > > > (Web page: http://groups.yahoo.com/gro
> > > > up/amiquote/messages/)
> > > > > > > > > > --------------------------------------------
> > > > > > > > > > Check group FAQ at:
> > > > > > > > > > http://groups.yahoo.com/group/amibroke
> > > > r/files/groupfaq.html
> > > > > > > > > >
> > > > > > > > > > Your use of Yahoo! Groups is subject to the Yahoo!
> Terms
> > > > of
> > > > > > > Service
> > > > > > > > > > <http://docs.yahoo.com/info/terms/> .
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Get A Free Psychic Reading! Your Online Answer To Life's Important Questions.
http://us.click.yahoo.com/Lj3uPC/Me7FAA/ySSFAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|