[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: Another tough question...



PureBytes Links

Trading Reference Links

> I agree that there is a serious problem when the only data that is 
>available
> contains no period that is similar to what is expected in the 
>future.
> 

Getting enough data is an issue for EOD traders.

A few possible solutions I have mentioned in the past (I like 'live' 
work but the negative is that it doesn't persist - unlike a book).

- new traders should work in old EOD data, say 1995-2000, until they 
address all of the basic issues, like length of IS versus OS etc.

They should save up the best years (the current 5) until they start 
to get good == backtests > 30-40% per annum on OOS tests and then 
move to fresh and/or bought data for confirmation/trading.

(of course we know that a lot of ideals will never make it to common 
practice - some are just too hard to sell).

- use other markets (that is why I highlighted the S&P global 1200 in 
a UKB post) - a US trader could practice on the ASX top 20 for 
example - ASX market behaviour of the 20 most liquid stocks is 
similar to the US top 100 or 200.

- become an intraday trader (plenty of data then)

- take a ten year history that included different market conditions, 
filter it for liquid stock (for concept testing I like only stock 
that trade everyday - no data holes - in real time I know when a 
stock isn't trading) - sort the data by 10 year performance i.e. 10 
year % return - assign them an ordinal number - then put every even 
stock in an IS testing watchlist and every odd stock in an OOS 
testing watchlist.

Now you have a 10 year IS and OOS database with a range of conditions 
and equal numbers of bullish and bearish stock.

I have done that with the most liquid stock in Jim's Yahoo database 
and I am comfortable working with it like that.

> Artificial data has no value.

One exception is for training.

I have learnt a lot using (crude) randomly generated data as a 
training benchmark - I regard the Black Swan as my adversary so I 
have studied his/her habits in depth.

The beauty of RGD is that, while it is not real, it is lifelike,and 
more importantly, we know in advance what it's real performance is 
(W/L ratio, %period returns).

I can't recommend that type of synthetic trading highly enough.

In all other trading tests we never ever have certainty about those 
numbers - I love the certainty of simulated data for comparing real 
behaviour to theoretical behaviour (if they don't mactch then I am 
not confident my theories will stand up in real life).


brian_z



--- In amibroker@xxxxxxxxxxxxxxx, "Howard B" <howardbandy@xxx> wrote:
>
> Hi Louis --
> 
> I agree that there is a serious problem when the only data that is 
available
> contains no period that is similar to what is expected in the 
future.
> 
> Artificial data has no value.
> 
> Using data that is earlier in time than the in-sample period has 
limited
> value.  You can test earlier data, but you will over-estimate the
> performance that you can expect in the future.
> 
> Are there other tickers that are closely related that have data for 
the
> periods you would like to test?
> 
> In the end, you will need to make a decision on whether to place 
actual
> trades.  And that decision must be based on your understanding of 
and
> confidence in your system.  The only way to gain that confidence is 
by
> observing the transitions from in-sample testing to out-of-sample 
simulated
> trading.
> 
> Thanks,
> Howard
> On Tue, Apr 8, 2008 at 10:37 PM, Mike <sfclimbers@xxx> wrote:
> 
> >   Howard's comments are consistent with those of Robert Pardo (The
> > Evaluation and Optimization of Trading Strategies, Wiley 2008), 
with
> > respect to training periods.
> >
> > Pardo recognizes that there is a tradeoff between more robust
> > strategies which require longer in sample training periods, 
require
> > fewer reoptimizations, trade for longer out of sample periods and 
are
> > generally less profitable, vs. more responsive strategies which
> > require shorter in sample training periods, require more frequent
> > reoptimizations, can only trade for shorter out of sample periods 
and
> > are generally more profitable.
> >
> > Pardo suggests that strategies generating more frequent signals 
can
> > use shorter in sample training windows since they generate the
> > minimum 30+ trades sooner than strategies that generate less 
frequent
> > signals. But, that in any case, one should try to use an in sample
> > period sufficiently long to capture bull, bear, and sideways 
markets.
> >
> > Further, when first trying to evaluate the worth of the strategy,
> > Pardo suggests backtesting the in sample history in segments 
rather
> > than one shot (e.g. 10 year history divided into five 2 year
> > segments). This gives you better insight as to whether the results
> > are due to a single segment or are consistent accross segments, 
and
> > provides insight to your eventual in sample/out of sample periods 
for
> > Walk Forward Optimization.
> >
> > Finally, Pardo suggests that regardless of whether a long or short
> > training period is used, a rule of thumb for in sample vs. out of
> > sample is for out of sample to be between 1/8 to 1/3 of the in 
sample
> > period (e.g. 24/8 = 3 and 24/3 = 8, so it would be "safe" to trade
> > out of sample for 3 - 8 months based on a system backtested over 
24
> > months.
> >
> > Yet another good book covering the topic. I reccomend it.
> >
> > Mike
> >
> >
> > --- In amibroker@xxxxxxxxxxxxxxx <amibroker%
40yahoogroups.com>, "Howard B"
> > <howardbandy@> wrote:
> > >
> > > Hi Louis, and all --
> > >
> > > I know David Aronson, respect him, and like and recommend his 
book.
> > >
> > > My view is that the in-sample period should be as short as
> > practical. My
> > > thought is that: the system we are testing / trading is trying 
to
> > recognize
> > > the signal from among the noise; and the signal patterns are
> > changing over
> > > time. So the length of the in-sample period is a tradeoff -- 
short
> > to be
> > > able to change as the characteristics of the underlying market
> > change, but
> > > not so short that the system is over-fit to the noise rather 
than
> > learns the
> > > signal.
> > >
> > > You can test this in AmiBroker. Have your system ready to buy 
and
> > sell. In
> > > the Automatic Analysis window, use Settings and set up the Walk
> > Forward
> > > parameters. Try an in-sample period of 10 years, an out-of-
sample
> > period of
> > > 6 months or 1 year. Run Optimize > Walk Forward and look at the 
in-
> > sample
> > > and out-of-sample equity curves. Shorten the length of the in-
> > sample period
> > > to 9, then 8, then 7, ... then 1 year, keeping the out-of-sample
> > period
> > > unchanged. Depending on your system and the market it is 
trading,
> > you may
> > > find that there is a sweet spot in the length of the in-sample
> > data. If so,
> > > that is the amount of data that allows your system to recognize 
the
> > signal
> > > without being overwhelmed by the noise.
> > >
> > > Thanks,
> > > Howard
> > >
> > >
> > > On Tue, Apr 8, 2008 at 8:56 AM, Louis Préfontaine <rockprog80@>
> >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I've been thinking a lot lately, and here is something I would
> > like to
> > > > have your opinion on.
> > > >
> > > > I've been introduced to automated systems by a trend following
> > book which
> > > > related how some trend followers built their systems in the 
70s
> > or 80s and
> > > > got rich with them, and how their system did not really change
> > all this
> > > > time. They didn't change their system because they say the
> > market does NOT
> > > > change. They looked at historic market data from the 1800s and
> > the market
> > > > was as it is right now. So they say.
> > > >
> > > > On the other side, lately I have been introduced to the 
concept of
> > > > ever-changing markets and have had a hard time trying to 
build my
> > system.
> > > > Got a very promising start with a system getting around 15-20%
> > average for
> > > > April 2007 to April 2008 (with little drawdown, which mean 
that
> > with
> > > > leverage I can boost this a lot). In any variation over
> > thousands of stocks
> > > > the results were nearly all positives. But then, I tested that
> > same system
> > > > for the years 2000 to 2008, and that was disappointing. Even 
more
> > > > disappointing from 2001 to 2003, another troubled market like 
the
> > one we are
> > > > in right now.
> > > >
> > > > So here I am, wondering where to go from now. Aronson's
> > excellent book
> > > > talk about the importance of having a very large sample of 
data.
> > But the
> > > > problem is: the larger the data, the more "historic" it gets 
and
> > the less it
> > > > seems to work.
> > > >
> > > > Is my system not working, or did the markets really change? 
Do I
> > need to
> > > > make it more robust (that is, it MUST make profit even from 
2001
> > to 2003),
> > > > or can I have complete faith in what happened in the last 
year?
> > > >
> > > > All those questions... Would be nice to read what you think
> > about this.
> > > >
> > > > Louis
> > > >
> > > >
> > >
> >
> >  
> >
>



------------------------------------

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/