|
PureBytes Links
Trading Reference Links
|
Brian,
In response to ...
... "Have you published a recommended method, or commentary, for these difficult analyses anywhere?
Has anyone else published anything on sensitivity analysis worth reading?
Thanks for the feedback ... sensitivity analysis is the aspect of opt that is leaping out at me." ...
Yes ... It's called IO ... ;) ... Whether or not you believe sensitivity analysis has any value during the IS optimization phase, if the topic is of interest to you I suggest you read that section of the IO doc which can be found in the files section here.
In particular:
- Pages 18 - 20 for the graphical presentation of sensitivity of variable values arrived at during optimization which can be utilized regardless of the number of variables
- Pages 36 - 41 for theory behind, methodology of and results from sensitivity testing during IS optimization
- Pages 34 - 35 for directives that can be used to affect how thoroughly sensitivity testing during IS optimization can be performed
I had posted some abbreviated content about this in the UKB but I don't know if thats still there as I don't seem to be able to reach the UKB url today. In any case the pages in the doc I referenced above have more diagrams and pictures than words so it shouldn't take long to get through what's there with the middle reference being the real guts.
----- Original Message -----
From: brian_z111
Date: Sunday, June 21, 2009 1:59 am
Subject: [amibroker] Re: Benchmarking
To: amibroker@xxxxxxxxxxxxxxx
> > This may be true for your brain, but not for a mechanical
> trading >system ...
>
>
> True, mechanical system analysis takes the subjective factor out
> of the equation, provided the practitioner doesn't inadvertently
> let subjective analysis creep in.
>
> Mechanical system analyses are performed by people with brains though.
>
> The value of any analysis, however, rests on the premise that
> the future will resemble the past with enough similarity to make
> historical data relevant ... if the future is changing faster
> than in the past where are we going to get the data to test?
>
> Of course a prediction that the future will not be similar to
> the past isn't worth the paper it is written on ... it doesn't
> hurt to be cautious though, especially when you see some warning
> signs, does it?
>
> >In more complex optimizations .... this a little more than a
> little >bit more difficult ..... where it counts i.e. during the
> >optimization process itself.
>
> Have you published a recommended method, or commentary, for
> these difficult analyses anywhere?
>
> Has anyone else published anything on sensitivity analysis worth
> reading?
> Thanks for the feedback ... sensitivity analysis is the aspect
> of opt that is leaping out at me.
>
> --- In amibroker@xxxxxxxxxxxxxxx, ftonetti@xxx wrote:
> >
> > Brian,
> >
> > In regards to your statement of ...
> >
> > ... "As an aside I am sceptical that there is evne such a
> thing as OOS when we use historical data ... for one thing we
> are all too widely read to be naive about any trading idea so in
> that sense we have all walked over any historical data, we care
> to get our hands on, thousands of times .. in that sense live
> trading is the only 'real' OutOfSample data" ...
> >
> > This may be true for your brain, but not for a mechanical
> trading system ...
> >
> > As far as optimization is concerned the way to work the
> problem is backwards ...
> >
> > Assuming that the goal is to have a robust set of variable
> values i.e. one that is most likely to do well OOS then this
> translates into having those parameter values not be highly
> sensitive. In a two variable system this is fairly easy to
> visualize with AB's 3d plots picking whatever performance metric
> one wants for the the z-axis. In more complex optimizations
> that involve more than 2 variables and/or enough combinations
> where exhaustive search is not feasible then this a little more
> than a little bit more difficult even after the fact let alone
> where it counts i.e. during the optimization process itself.
> >
>
>
>
__._,_.___
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
![]()
__,_._,___
|