|
PureBytes Links
Trading Reference Links
|
Bill,
Yes, I see there is some good discussion 'back there' that can be
found by searching for the well known authors - thanks for that.
SDT&E is a big mountain - I haven't climbed it yet.
I am just sitting at my present level for a while.
I have reached the point where I don't read every line in all of
those books - I only quickly flick read them looking for a point of
difference, and I don't really find any - all the commentators are
circling around the same theme (the exception would be Aronson, and
the people in or around that circle, who did the work on MCP - I
thought that was a break from tradition, and a valiant effort).
Personally, I am going in a completely different direction and I am
comfortable with the results I get.
I didn't ever set out with the intention of writing a book but as I
said I have enough interesting and unique material already to suggest
that one is possible - it's just not mature enough at this stage and
it would involve a mammoth effort, on my part, from here (I don't
actually need it for my own trading - it would only benefit others).
The point is that I want to put it OTR that walk-forward IS/OOS is
not the only game in town.
brian_z111
--- In amibroker@xxxxxxxxxxxxxxx, "wavemechanic" <timesarrow@xxx>
wrote:
>
> I would add one thing - the out of sample test should be the same
type of market as that used for the in sample test. I have seen, for
example, some good neural net models that take this into account,
resulting in models for up, down, and flat markets, as opposed to one
model fits all. Of course, defining up, down, and flat can be a bit
subjective but performance definitely improves when one takes a shot
at it. In any case, the equity curve (bottom line) will keep one
generally on the right track without serious derailments. There was
much discussion of this topic on this board several years ago with
reference to the many books and articles on the subject which all
tend to come down in the same place.
>
> Bill
>
> ----- Original Message -----
> From: Howard B
> To: amibroker@xxxxxxxxxxxxxxx
> Sent: Tuesday, February 05, 2008 10:38 AM
> Subject: Re: [amibroker] What is a valid number of Back test
results to Optimize?
>
>
> Hi Chris --
>
> You can do anything you want to in your search for a good trading
system. The data period you work with during that search is the in-
sample period. The results you achieve over the in-sample period
have no value in predicting what the future performance will be. In
order to estimate the future performance, you need to test the
program on a set of data that follows the in-sample period and has
not been used at all in the development of the system. That data is
called the out-of-sample data. You can perform statistical tests on
the out-of-sample results, but the quickest way to evaluate it is to
look at the out-of-sample equity curve.
>
> Be careful to avoid the following procedure. Optimize in-sample,
evaluate out-of-sample, modify the system based on the the out-of-
sample results, retest out-of-sample. The previously out-of-sample
data period has become part of an expanded in-sample data set and a
new out-of-sample test is required in order to estimate future
performance.
>
> There is a lot more to system development, testing, and
validation than those two paragraphs. I am presenting a two-day
workshop in Las Vegas February 21 and 22 devoted to that subject.
> http://www.ftmonitor.com/lv08/lv08intro.html
>
> And I have written a book devoted to that subject.
> http://www.quantitativetradingsystems.com/
>
> Thanks,
> Howard
>
>
> On Feb 5, 2008 4:34 AM, ChrisB <kris45mar@xxx> wrote:
>
> What is a valid or reasonable number of backtest results to
subject to
> Optimization?
>
> For general statistics a minimum of 30 or so is needed to start
getting
> valid StdDevs etc.
>
> If I run a backtest on hourly currency data over three months I
get
> around 16 -20 tradeable signals per currency.
> This give a nice smooth plateau on 3D optimization.
>
> If I test over two months of data I get around 10 - 12 trades
>
> If I test over only 1 month I get only 5 or 6 trades.
>
> These shorter time periods still give visually acceptable 3D
plateaus
> but I am wondering if there is enough data to be statistically
significant.
>
> I am trying to get a handle on how close I can get to current
> fluctuations in the market without hitting noise. The idea
being to redo
> the Optimization every x time frame and shift the entry and
exit
> parameters to stay in the middle of the plateau.
>
> Of course I can backtest over longer time frames, say 6 months
of data,
> shifting the starting date forward by one month at a time, but
this
> would seem to introduce more "lag" into my selection of best
parameters
> to trade.
>
> Does anyone have any thoughts/references on this?
>
> --
> Regards
>
> ChrisB
>
>
>
>
>
>
>
> --------------------------------------------------------------------
----------
>
>
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.5.516 / Virus Database: 269.19.19/1258 - Release Date:
2/4/2008 10:10 AM
>
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|