|
PureBytes Links
Trading Reference Links
|
Hi.
I was following your conversation while I was on a trip in Malaysia. I tried
to contribute a few times but I am not sure wether you received my mails. I
found the walk forward optimization very interesting and think it is one of
the ways to see wether optimization makes sence at all.
I tried to convince one of our software users to work and publish a few more
systems on that matter, but unfortunatly he is busy with other systems (or
he does not want to for other reason ;)).
Anyway, this guy has created a system based on RSI that does what you are
discussing here - walk forward optimization.
Since I am not a programmer I am not able to test/programm this feature, but
the available script could be used by you as an base. We could then REALLY
test and see wether WFO makes sence.
Here is the script:
http://www.wealth-lab.com/cgi-bin/WealthLab.DLL/editsystem?id=13406
http://www.wealth-lab.com/cgi-bin/WealthLab.DLL/editsystem?id=4288
I just saw that Willibald has published a new version of it..... very good.
This could be the base of your/our discussion here.
Willibald, are you in this discussion?
Regards.
Volker Knapp
Wealth-Lab Inc.
http://www.wealth-lab.de
http://www.wealth-lab.com
--- Leslie Walko <l.walko@xxxxxxxxxxx> schrieb: > Datum: Fri, 01 Nov 2002
10:16:19 -0500
> Von: Leslie Walko <l.walko@xxxxxxxxxxx>
> An: Jim Johnson <jejohn@xxxxxxxxxxx>
> CC: omega-list@xxxxxxxxxx, mark.keenan@xxxxxxxxxxxxxx
> Betreff: Re: Rolling Window Forward Optimization or Anchored
> ForwardOptimization
> - System results
>
> Jim:
>
> Dennis Miller is correct in noting the amount of work involved.
> Large.
> However, even if you put the work in, what are you testing?
> Two different systems, not optimization efficacy.
> Why?
>
> System one:
> Two linear regression values, each optimized over
> 5,000 bars and trading signals taken on cross over / under.
> Simple.
> System one carries an optimization load of 2
> (2 degrees of freedom) for every 5,000 bars.
>
> System two:
> Same logic, but divided into 1,000 bar segments, the rolling
> window size. Therefore, you have 2 degrees of freedom for
> every window. The parameter to sample size load is 5 times
> greater.
>
> System two MIGHT perform better or it might perform worse
> than system one depending on how well the 1,000 bar window
> matches the resonance frequency of the underlying data set.
>
> Conclusion: the proposed method for rolling window
> optimization does NOT discriminate between the efficacy of
> one kind of optimization versus another, but discriminates
> between a 1st order polynomial curve fit versus a higher
> order polynomial curve fit system.
> A serious methodological error.
>
> For further discussion, see Tuchar Chande: Beyond Technical
> Analysis, Second Edition; Wiley & Sons, 2001; pages 44 - 48.
>
> Regards,
> Leslie
>
>
>
> Jim Johnson wrote:
> >
> > Hello Leslie,
> >
> > As Dennis Miller says, "I could be wrong but..." it would be a lot of
> > work but it seems you could test your walk forward method directly
> > rather than rely on assumptions. As an example:
> >
> > optimize a system on 1990 data.
> >
> > check its results on 1991 data.
> >
> > optimize on 1991.
> >
> > check its results on 1992.
> >
> > etc etc.
> >
> > The results from "checking" would give you the actual performance of
> > the "system"--system now defined as trading rules plus
> > re-optimization.
> >
> > Best regards,
> > Jim Johnson mailto:jejohn@xxxxxxxxxxx
> >
> > --
> > Thursday, October 31, 2002, 10:56:33 PM, you wrote:
> >
> > LW> Mark:
> > LW> Perhaps if you rethink your question more in terms of 'what is
> > LW> the hypothesis that I am attempting to test', you might find
> > LW> more enlightenment than if you were searching for
> > LW> "producing more reliable results".
> >
> > LW> The theory behind walk forward optimization is that the market
> > LW> inefficiency you plan to trade in the near future has existed
> > LW> only for limited periods in the past and may manifest themselves
> > LW> for only a little while longer. Certain markets, say SP day
> > LW> trade SEEM to display this characteristic.
> >
> > LW> If this theory were true in a strict sense then rolling window
> > LW> would give you the best results assuming that (a) you selected
> > LW> the in sample size correctly; and (b) you selected the out of
> > LW> sample size correctly. This might a a bit hard to do :-)
> >
> > LW> A weaker version of this theory is that DUH something happened
> > LW> that changed the market and the 'old' market efficiency was
> > LW> replaced by the 'new' market inefficiency that you are going to
> > LW> get rich on -- assuming you find the magic DUH moment
> > LW> to begin your in sample optimization at the right time.
> >
> > LW> Notice that both the strong and the weak form of the walk forward
> > LW> optimization hypothesis contain untestable auxiliary hypothesis
> > LW> about beginning periods, in sample, and out of sample size.
> >
> > LW> So, it this a 'good theory'?
> > LW> A philospher of science would have serious reservations about it.
> > LW> A philosopher would want to know how your system perform
> > LW> using 'regular' optimization versus 'walk forward optimization'
> > LW> using TRUE OUT OF SAMPLE DATA.
> > LW> For the philosopher the 'goodness of a theory' is relative to
> > LW> the goodness of other theories.
> > LW> If you cannot even determine how two theories compare to each
> > LW> other, a philosopher might advise you to abandon your useless
> > LW> theory, (or write a book about it to collect royalties).
> >
> > LW> Enjoy,
> > LW> Leslie
> >
> > LW> mark.keenan@xxxxxxxxxxxxxx wrote:
> > >>
> > >> Any opinions or experience on whether anchored forward optimization
> > >> (keeping the start date fixed) produces more reliable results than
> the more
> > >> common walk forward optimization, where the "in sample window is
> rolled
> > >> forward"
> > >>
> > >> Most walk forward examples in books describe optimizing over 2
> years for
> > >> example - trading the optimized parameters over the next six months
> - and
> > >> the rolling the whole two year window forward by six months and the
> > >> repeating.
> > >>
> > >> My own testing has found anchored to be much more effective - I
> guess
> > >> building on the concept that if your going to optimize anyway then
> use as
> > >> much data as possible.
> > >>
> > >> I have been doing some work on a Double Linear Regression slope
> system on
> > >> index futures where I optimize over the whole data range (3 years -
> as
> > >> before three years the market was traded on the floor - is now
> screen
> > >> traded for the last 3 years) trade the result on the next month and
> then
> > >> add this recent "traded" month to the data series - re-optimize,
> trade on
> > >> the following month etc. Therefore as time passes my in sample data
> length
> > >> keeps getting bigger and bigger.
> > >>
> > >> On OUT OF SAMPLE trading - WITHOUT any stops yet - still doing the
> MAE
> > >> analysis the system is showing the following results using anchored
> > >> analysis.
> > >>
> > >> Total Trades 143
> > >> % Winning 51.05%
> > >>
> > >> Average Losing 2.95%
> > >> Average Winning 3.35%
> > >> Average Trade 0.31%
> > >>
> > >> Largest Winner 18.63%
> > >> Largest Loser 10.6% (would obviously be stopped out as
> below)
> > >>
> > >> Profit Factor 1.27
> > >> MAE average 2.31% (will be re-running results with a
> 2.50% stop
> > >> I think)
> > >>
> > >> Any views on the above??
> > >>
> > >> MK
> > >>
> > >> (sorry about disclaimer)
> > >>
> > >>
>
---------------------------------------------------------------------------
> > >> This message (including any attachments) is confidential and may be
> > >> privileged. If you have received it by mistake please notify the
> sender by
> > >> return e-mail and delete this message from your system. Any
> unauthorised
> > >> use or dissemination of this message in whole or in part is
> strictly
> > >> prohibited. Please note that e-mails are susceptible to change.
> > >> ABN AMRO Bank N.V. (including its group companies) shall not be
> liable for
> > >> the improper or incomplete transmission of the information
> contained in
> > >> this communication nor for any delay in its receipt or damage to
> your
> > >> system. ABN AMRO Bank N.V. (or its group companies) does not
> guarantee that
> > >> the integrity of this communication has been maintained nor that
> this
> > >> communication is free of viruses, interceptions or interference.
> > >>
>
---------------------------------------------------------------------------
>
> --
> Regards,
> Leslie Walko
> 610-688-2442
> --
> "Life is a tragedy for those who feel, a comedy for those who
> think"
> Horace Walpole, 4th earl of Orford, in a letter dated about 1770
>
|