|
PureBytes Links
Trading Reference Links
|
Hi everyone,
I am not at home right now, and it's really a pleasure to read you
while drinking this marvelous Côtes-du-Rhônes. I really appreciate
all the ideas you shared with me (and the group).
I must say that everyone seem to have different visions of the
problem, with people focusing on walk-forward optimizations, others
on specific date backtesting and still others (someone who contacted
me in private) refuse to backtest and want to trade directly with
easy to follow rules.
For what I have read (and I will re-read tomorrow, at home), I need
more data if I want to follow the « rule » of the 30 trades. Right
now, my system is based on a major index and it issues only about 3-
5 signals a year (which, at a 20% portfolio ratio is 15 to 25
trades), so I would need between 6 to 10 years of data, which of
course is impossible to do because we all know the market 10 years
ago as nothing to do with what it is right now.
On the other side, I could use minimal backtesting, but then the
data-mining bias would increase, considering that my system has only
a very limited of trades each year. Let's say that if I use only
one year back-testing (a bull market, a sideway market, and a bear
to sideway market), that would be about 3-5 trades. How can I say
with certitude that the gains are not due to luck on such a small
amount of signals on the major index? Even if I get 30 trades from
buying the stocks linked to the index, this may still be only data-
mining to the major index, as the stocks tend to follow that index.
(As an example, if I data-mine perfectly the Dow Jones, chances are
that buying the 30 companies in the index will give a good
result... I would have a lot more trades, but in fact they would be
based only on the same data-mined Dow Jones index...) --BIG PROBLEM-
-
Finally, there is the suggestion of going intraday. I'd like to do
this, butmy RT data provider only provides 1 year of intraday data.
Do you know other provider who gives more?
And finally finally... Are you sure Brian one can expect 40% per
annumm on EOD data? This seem like very very good!
My strategy right now is based on a very limited number of trades
with extra-limited drawdowns (I need to thank a member of this board
who helped me with this... You know who you are... Thanks again!).
So I can put maximum margin and boost the results. But with extra-
little trades comes the problem of significance of the results: are
the results good because the system is good or are the results good
be cause of good luck?
That was the purpose of the first message, and so far I have new
ideas but I am still wondering what I should do.
Louis
--- In amibroker@xxxxxxxxxxxxxxx, "brian_z111" <brian_z111@xxx>
wrote:
>
> > I agree that there is a serious problem when the only data that
is
> >available
> > contains no period that is similar to what is expected in the
> >future.
> >
>
> Getting enough data is an issue for EOD traders.
>
> A few possible solutions I have mentioned in the past (I
like 'live'
> work but the negative is that it doesn't persist - unlike a book).
>
> - new traders should work in old EOD data, say 1995-2000, until
they
> address all of the basic issues, like length of IS versus OS etc.
>
> They should save up the best years (the current 5) until they
start
> to get good == backtests > 30-40% per annum on OOS tests and then
> move to fresh and/or bought data for confirmation/trading.
>
> (of course we know that a lot of ideals will never make it to
common
> practice - some are just too hard to sell).
>
> - use other markets (that is why I highlighted the S&P global 1200
in
> a UKB post) - a US trader could practice on the ASX top 20 for
> example - ASX market behaviour of the 20 most liquid stocks is
> similar to the US top 100 or 200.
>
> - become an intraday trader (plenty of data then)
>
> - take a ten year history that included different market
conditions,
> filter it for liquid stock (for concept testing I like only stock
> that trade everyday - no data holes - in real time I know when a
> stock isn't trading) - sort the data by 10 year performance i.e.
10
> year % return - assign them an ordinal number - then put every
even
> stock in an IS testing watchlist and every odd stock in an OOS
> testing watchlist.
>
> Now you have a 10 year IS and OOS database with a range of
conditions
> and equal numbers of bullish and bearish stock.
>
> I have done that with the most liquid stock in Jim's Yahoo
database
> and I am comfortable working with it like that.
>
> > Artificial data has no value.
>
> One exception is for training.
>
> I have learnt a lot using (crude) randomly generated data as a
> training benchmark - I regard the Black Swan as my adversary so I
> have studied his/her habits in depth.
>
> The beauty of RGD is that, while it is not real, it is
lifelike,and
> more importantly, we know in advance what it's real performance is
> (W/L ratio, %period returns).
>
> I can't recommend that type of synthetic trading highly enough.
>
> In all other trading tests we never ever have certainty about
those
> numbers - I love the certainty of simulated data for comparing
real
> behaviour to theoretical behaviour (if they don't mactch then I am
> not confident my theories will stand up in real life).
>
>
> brian_z
>
>
>
> --- In amibroker@xxxxxxxxxxxxxxx, "Howard B" <howardbandy@> wrote:
> >
> > Hi Louis --
> >
> > I agree that there is a serious problem when the only data that
is
> available
> > contains no period that is similar to what is expected in the
> future.
> >
> > Artificial data has no value.
> >
> > Using data that is earlier in time than the in-sample period has
> limited
> > value. You can test earlier data, but you will over-estimate the
> > performance that you can expect in the future.
> >
> > Are there other tickers that are closely related that have data
for
> the
> > periods you would like to test?
> >
> > In the end, you will need to make a decision on whether to place
> actual
> > trades. And that decision must be based on your understanding
of
> and
> > confidence in your system. The only way to gain that confidence
is
> by
> > observing the transitions from in-sample testing to out-of-
sample
> simulated
> > trading.
> >
> > Thanks,
> > Howard
> > On Tue, Apr 8, 2008 at 10:37 PM, Mike <sfclimbers@> wrote:
> >
> > > Howard's comments are consistent with those of Robert Pardo
(The
> > > Evaluation and Optimization of Trading Strategies, Wiley
2008),
> with
> > > respect to training periods.
> > >
> > > Pardo recognizes that there is a tradeoff between more robust
> > > strategies which require longer in sample training periods,
> require
> > > fewer reoptimizations, trade for longer out of sample periods
and
> are
> > > generally less profitable, vs. more responsive strategies which
> > > require shorter in sample training periods, require more
frequent
> > > reoptimizations, can only trade for shorter out of sample
periods
> and
> > > are generally more profitable.
> > >
> > > Pardo suggests that strategies generating more frequent
signals
> can
> > > use shorter in sample training windows since they generate the
> > > minimum 30+ trades sooner than strategies that generate less
> frequent
> > > signals. But, that in any case, one should try to use an in
sample
> > > period sufficiently long to capture bull, bear, and sideways
> markets.
> > >
> > > Further, when first trying to evaluate the worth of the
strategy,
> > > Pardo suggests backtesting the in sample history in segments
> rather
> > > than one shot (e.g. 10 year history divided into five 2 year
> > > segments). This gives you better insight as to whether the
results
> > > are due to a single segment or are consistent accross
segments,
> and
> > > provides insight to your eventual in sample/out of sample
periods
> for
> > > Walk Forward Optimization.
> > >
> > > Finally, Pardo suggests that regardless of whether a long or
short
> > > training period is used, a rule of thumb for in sample vs. out
of
> > > sample is for out of sample to be between 1/8 to 1/3 of the in
> sample
> > > period (e.g. 24/8 = 3 and 24/3 = 8, so it would be "safe" to
trade
> > > out of sample for 3 - 8 months based on a system backtested
over
> 24
> > > months.
> > >
> > > Yet another good book covering the topic. I reccomend it.
> > >
> > > Mike
> > >
> > >
> > > --- In amibroker@xxxxxxxxxxxxxxx <amibroker%
> 40yahoogroups.com>, "Howard B"
> > > <howardbandy@> wrote:
> > > >
> > > > Hi Louis, and all --
> > > >
> > > > I know David Aronson, respect him, and like and recommend
his
> book.
> > > >
> > > > My view is that the in-sample period should be as short as
> > > practical. My
> > > > thought is that: the system we are testing / trading is
trying
> to
> > > recognize
> > > > the signal from among the noise; and the signal patterns are
> > > changing over
> > > > time. So the length of the in-sample period is a tradeoff --
> short
> > > to be
> > > > able to change as the characteristics of the underlying
market
> > > change, but
> > > > not so short that the system is over-fit to the noise rather
> than
> > > learns the
> > > > signal.
> > > >
> > > > You can test this in AmiBroker. Have your system ready to
buy
> and
> > > sell. In
> > > > the Automatic Analysis window, use Settings and set up the
Walk
> > > Forward
> > > > parameters. Try an in-sample period of 10 years, an out-of-
> sample
> > > period of
> > > > 6 months or 1 year. Run Optimize > Walk Forward and look at
the
> in-
> > > sample
> > > > and out-of-sample equity curves. Shorten the length of the
in-
> > > sample period
> > > > to 9, then 8, then 7, ... then 1 year, keeping the out-of-
sample
> > > period
> > > > unchanged. Depending on your system and the market it is
> trading,
> > > you may
> > > > find that there is a sweet spot in the length of the in-
sample
> > > data. If so,
> > > > that is the amount of data that allows your system to
recognize
> the
> > > signal
> > > > without being overwhelmed by the noise.
> > > >
> > > > Thanks,
> > > > Howard
> > > >
> > > >
> > > > On Tue, Apr 8, 2008 at 8:56 AM, Louis Préfontaine
<rockprog80@>
> > >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I've been thinking a lot lately, and here is something I
would
> > > like to
> > > > > have your opinion on.
> > > > >
> > > > > I've been introduced to automated systems by a trend
following
> > > book which
> > > > > related how some trend followers built their systems in
the
> 70s
> > > or 80s and
> > > > > got rich with them, and how their system did not really
change
> > > all this
> > > > > time. They didn't change their system because they say the
> > > market does NOT
> > > > > change. They looked at historic market data from the 1800s
and
> > > the market
> > > > > was as it is right now. So they say.
> > > > >
> > > > > On the other side, lately I have been introduced to the
> concept of
> > > > > ever-changing markets and have had a hard time trying to
> build my
> > > system.
> > > > > Got a very promising start with a system getting around 15-
20%
> > > average for
> > > > > April 2007 to April 2008 (with little drawdown, which mean
> that
> > > with
> > > > > leverage I can boost this a lot). In any variation over
> > > thousands of stocks
> > > > > the results were nearly all positives. But then, I tested
that
> > > same system
> > > > > for the years 2000 to 2008, and that was disappointing.
Even
> more
> > > > > disappointing from 2001 to 2003, another troubled market
like
> the
> > > one we are
> > > > > in right now.
> > > > >
> > > > > So here I am, wondering where to go from now. Aronson's
> > > excellent book
> > > > > talk about the importance of having a very large sample of
> data.
> > > But the
> > > > > problem is: the larger the data, the more "historic" it
gets
> and
> > > the less it
> > > > > seems to work.
> > > > >
> > > > > Is my system not working, or did the markets really
change?
> Do I
> > > need to
> > > > > make it more robust (that is, it MUST make profit even
from
> 2001
> > > to 2003),
> > > > > or can I have complete faith in what happened in the last
> year?
> > > > >
> > > > > All those questions... Would be nice to read what you think
> > > about this.
> > > > >
> > > > > Louis
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> >
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|