|
PureBytes Links
Trading Reference Links
|
Volker:
I've already discussed the main issues on this, but just a
pointer.
You cannot test the efficacy of walk forward optimization on
an indicator like RSI because the RSI, by design is a medium
pass filter. Any trend that last about twice or longer than
the sampling window of the RSI will put the RSI value to
saturation levels. In other words, the (original) 30 - 70
lines become useless.
The only thing that your 'walk forward test' would tell you
is that, DUH, a medium pass filter can be 'improved' by
setting the sample size to 1/2 of the dominant cycle
length. I think I already know that and do not need a
complex mousetrap to test it.
Best regards,
Leslie
VK wrote:
>
> Hi.
>
> I was following your conversation while I was on a trip in Malaysia. I tried
> to contribute a few times but I am not sure wether you received my mails. I
> found the walk forward optimization very interesting and think it is one of
> the ways to see wether optimization makes sence at all.
>
> I tried to convince one of our software users to work and publish a few more
> systems on that matter, but unfortunatly he is busy with other systems (or
> he does not want to for other reason ;)).
>
> Anyway, this guy has created a system based on RSI that does what you are
> discussing here - walk forward optimization.
>
> Since I am not a programmer I am not able to test/programm this feature, but
> the available script could be used by you as an base. We could then REALLY
> test and see wether WFO makes sence.
>
> Here is the script:
>
> http://www.wealth-lab.com/cgi-bin/WealthLab.DLL/editsystem?id=13406
>
> http://www.wealth-lab.com/cgi-bin/WealthLab.DLL/editsystem?id=4288
>
> I just saw that Willibald has published a new version of it..... very good.
> This could be the base of your/our discussion here.
>
> Willibald, are you in this discussion?
>
> Regards.
>
> Volker Knapp
> Wealth-Lab Inc.
> http://www.wealth-lab.de
> http://www.wealth-lab.com
>
> --- Leslie Walko <l.walko@xxxxxxxxxxx> schrieb: > Datum: Fri, 01 Nov 2002
> 10:16:19 -0500
> > Von: Leslie Walko <l.walko@xxxxxxxxxxx>
> > An: Jim Johnson <jejohn@xxxxxxxxxxx>
> > CC: omega-list@xxxxxxxxxx, mark.keenan@xxxxxxxxxxxxxx
> > Betreff: Re: Rolling Window Forward Optimization or Anchored
> > ForwardOptimization
> > - System results
> >
> > Jim:
> >
> > Dennis Miller is correct in noting the amount of work involved.
> > Large.
> > However, even if you put the work in, what are you testing?
> > Two different systems, not optimization efficacy.
> > Why?
> >
> > System one:
> > Two linear regression values, each optimized over
> > 5,000 bars and trading signals taken on cross over / under.
> > Simple.
> > System one carries an optimization load of 2
> > (2 degrees of freedom) for every 5,000 bars.
> >
> > System two:
> > Same logic, but divided into 1,000 bar segments, the rolling
> > window size. Therefore, you have 2 degrees of freedom for
> > every window. The parameter to sample size load is 5 times
> > greater.
> >
> > System two MIGHT perform better or it might perform worse
> > than system one depending on how well the 1,000 bar window
> > matches the resonance frequency of the underlying data set.
> >
> > Conclusion: the proposed method for rolling window
> > optimization does NOT discriminate between the efficacy of
> > one kind of optimization versus another, but discriminates
> > between a 1st order polynomial curve fit versus a higher
> > order polynomial curve fit system.
> > A serious methodological error.
> >
> > For further discussion, see Tuchar Chande: Beyond Technical
> > Analysis, Second Edition; Wiley & Sons, 2001; pages 44 - 48.
> >
> > Regards,
> > Leslie
> >
> >
> >
> > Jim Johnson wrote:
> > >
> > > Hello Leslie,
> > >
> > > As Dennis Miller says, "I could be wrong but..." it would be a lot of
> > > work but it seems you could test your walk forward method directly
> > > rather than rely on assumptions. As an example:
> > >
> > > optimize a system on 1990 data.
> > >
> > > check its results on 1991 data.
> > >
> > > optimize on 1991.
> > >
> > > check its results on 1992.
> > >
> > > etc etc.
> > >
> > > The results from "checking" would give you the actual performance of
> > > the "system"--system now defined as trading rules plus
> > > re-optimization.
> > >
> > > Best regards,
> > > Jim Johnson mailto:jejohn@xxxxxxxxxxx
> > >
> > > --
> > > Thursday, October 31, 2002, 10:56:33 PM, you wrote:
> > >
> > > LW> Mark:
> > > LW> Perhaps if you rethink your question more in terms of 'what is
> > > LW> the hypothesis that I am attempting to test', you might find
> > > LW> more enlightenment than if you were searching for
> > > LW> "producing more reliable results".
> > >
> > > LW> The theory behind walk forward optimization is that the market
> > > LW> inefficiency you plan to trade in the near future has existed
> > > LW> only for limited periods in the past and may manifest themselves
> > > LW> for only a little while longer. Certain markets, say SP day
> > > LW> trade SEEM to display this characteristic.
> > >
> > > LW> If this theory were true in a strict sense then rolling window
> > > LW> would give you the best results assuming that (a) you selected
> > > LW> the in sample size correctly; and (b) you selected the out of
> > > LW> sample size correctly. This might a a bit hard to do :-)
> > >
> > > LW> A weaker version of this theory is that DUH something happened
> > > LW> that changed the market and the 'old' market efficiency was
> > > LW> replaced by the 'new' market inefficiency that you are going to
> > > LW> get rich on -- assuming you find the magic DUH moment
> > > LW> to begin your in sample optimization at the right time.
> > >
> > > LW> Notice that both the strong and the weak form of the walk forward
> > > LW> optimization hypothesis contain untestable auxiliary hypothesis
> > > LW> about beginning periods, in sample, and out of sample size.
> > >
> > > LW> So, it this a 'good theory'?
> > > LW> A philospher of science would have serious reservations about it.
> > > LW> A philosopher would want to know how your system perform
> > > LW> using 'regular' optimization versus 'walk forward optimization'
> > > LW> using TRUE OUT OF SAMPLE DATA.
> > > LW> For the philosopher the 'goodness of a theory' is relative to
> > > LW> the goodness of other theories.
> > > LW> If you cannot even determine how two theories compare to each
> > > LW> other, a philosopher might advise you to abandon your useless
> > > LW> theory, (or write a book about it to collect royalties).
> > >
> > > LW> Enjoy,
> > > LW> Leslie
> > >
> > > LW> mark.keenan@xxxxxxxxxxxxxx wrote:
> > > >>
> > > >> Any opinions or experience on whether anchored forward optimization
> > > >> (keeping the start date fixed) produces more reliable results than
> > the more
> > > >> common walk forward optimization, where the "in sample window is
> > rolled
> > > >> forward"
> > > >>
> > > >> Most walk forward examples in books describe optimizing over 2
> > years for
> > > >> example - trading the optimized parameters over the next six months
> > - and
> > > >> the rolling the whole two year window forward by six months and the
> > > >> repeating.
> > > >>
> > > >> My own testing has found anchored to be much more effective - I
> > guess
> > > >> building on the concept that if your going to optimize anyway then
> > use as
> > > >> much data as possible.
> > > >>
> > > >> I have been doing some work on a Double Linear Regression slope
> > system on
> > > >> index futures where I optimize over the whole data range (3 years -
> > as
> > > >> before three years the market was traded on the floor - is now
> > screen
> > > >> traded for the last 3 years) trade the result on the next month and
> > then
> > > >> add this recent "traded" month to the data series - re-optimize,
> > trade on
> > > >> the following month etc. Therefore as time passes my in sample data
> > length
> > > >> keeps getting bigger and bigger.
> > > >>
> > > >> On OUT OF SAMPLE trading - WITHOUT any stops yet - still doing the
> > MAE
> > > >> analysis the system is showing the following results using anchored
> > > >> analysis.
> > > >>
> > > >> Total Trades 143
> > > >> % Winning 51.05%
> > > >>
> > > >> Average Losing 2.95%
> > > >> Average Winning 3.35%
> > > >> Average Trade 0.31%
> > > >>
> > > >> Largest Winner 18.63%
> > > >> Largest Loser 10.6% (would obviously be stopped out as
> > below)
> > > >>
> > > >> Profit Factor 1.27
> > > >> MAE average 2.31% (will be re-running results with a
> > 2.50% stop
> > > >> I think)
> > > >>
> > > >> Any views on the above??
> > > >>
> > > >> MK
> > > >>
> > > >> (sorry about disclaimer)
> > > >>
> > > >>
> >
> ---------------------------------------------------------------------------
> > > >> This message (including any attachments) is confidential and may be
> > > >> privileged. If you have received it by mistake please notify the
> > sender by
> > > >> return e-mail and delete this message from your system. Any
> > unauthorised
> > > >> use or dissemination of this message in whole or in part is
> > strictly
> > > >> prohibited. Please note that e-mails are susceptible to change.
> > > >> ABN AMRO Bank N.V. (including its group companies) shall not be
> > liable for
> > > >> the improper or incomplete transmission of the information
> > contained in
> > > >> this communication nor for any delay in its receipt or damage to
> > your
> > > >> system. ABN AMRO Bank N.V. (or its group companies) does not
> > guarantee that
> > > >> the integrity of this communication has been maintained nor that
> > this
> > > >> communication is free of viruses, interceptions or interference.
> > > >>
> >
> ---------------------------------------------------------------------------
> >
> > --
> > Regards,
> > Leslie Walko
> > 610-688-2442
> > --
> > "Life is a tragedy for those who feel, a comedy for those who
> > think"
> > Horace Walpole, 4th earl of Orford, in a letter dated about 1770
> >
>
> ------------------------------------------------------------
> Name: winmail.dat
> winmail.dat Type: application/ms-tnef
> Encoding: base64
--
Regards,
Leslie Walko
610-688-2442
--
"Life is a tragedy for those who feel, a comedy for those
who think"
Horace Walpole, 4th earl of Orford, in a letter dated about
1770
|