|
PureBytes Links
Trading Reference Links
|
Sursod,
Didn't dig too deep into your post due to time constraints, but I
feel you might better use your time to focus on testing in-sample
data results on randomly selected out-of-sample (OOS) data, and also
using a Monte Carlo Simulation (MCS) to determine if your final opt
settings are going to work. If you do minimal optimization runs and
find that the settings work well in random OOS and MCS, it won't
matter how much in-sample data you used previously. You will have
already reached your goal.
~Brian
--- In amibroker@xxxxxxxxxxxxxxx, "sursod" <sursod@xxx> wrote:
>
> I find "degree of freedom" as applied to trading system
development
> difficult to grasp and use (Robert Pardo, "Design, Testing, and
> Optimization of Trading Systems" ) .
> Insights please!
>
> Pardo wrote statisticians use the term degree of freedom (df) as a
> measurement of confidence in the test reults. Fewer specifications
> and more data give greater confidence but there is no formula that
> expresses the relationship (written in 1992, does a formula exist
> now?) Pardo made some suggestions:
>
> Degree of freedom = Number of signals – Number of rules,
> or
> 90% degree of freedom (factor of 10):
> Minumum df = 10 x (Rules + Conditions)
>
> He wrote a 30 day moving average should be tested on at least 300
> days of data (factor of 10) though satisfying the minimum
> requirements is not as good as using fewer rules and more data to
> produce a large number of trades ( P.56). Further on (P.140) he
> mentioned a system performing 980 tests (980 optimization
> combinations) on 1250 data points does not give confidence. My
> question - what factor to apply on the number of optimization
tests
> to give an adequate optimization period, a factor of 10 is
difficult
> to uphold since even simple systems run up a large niumber of
tests.
>
> Take a simple MACD eod system with buy and sell using the same
> settings. If we optimize using 10 steps for the "slow", "fast"
> and "sig" moving averages we have 10x10x10 = 1000 combinations
even
> without profit or loss stops. A factor of 10 would require
10,000
> days or close to 40 years of history for this simple system. A
> trading system development tutorial I found in the Amibroker
support
> zone website optimizes 7x9x21x11x21x50x50 steps (smoothing, CCI,
> lookback, Blevel, SLevel, stop, stop) which gives 764,032,500
tests.
>
> These examples make me doubt my understanding of the factor of 10
> rule, but factor or no factor it is still true the more we
optimize
> the larger the test window.
>
> Going back to the MACD example, suppose the 3 moving averages are
> optimized from
> 12 to 21 in steps of 1 = 10 tests
> 18 to 27 in steps of 1 = 10 tests
> 4 to 12 in steps of 1 = 10 tests
>
> Can we say they use up 30 degrees of freedom? Since we do not
count
> overlaps (based on an example Pardo gave)and there are 24 steps
from
> 4 to 27 can we say this system uses 24 df? In this case would the
> 90% df rule require a test sample to yield approx. 240 to 300
trade
> signals for statistical confidence? Whatever matrix we use the
test
> window should naturally also be large enough to cover enough
market
> conditions etc.
>
> My next question – suppose my buy rule is:
> Buy = (A and B and (C or D) and (E or F);
> The possible permutations:
> A and B and C and E
> A and B and C and F
> A and B and D and E
> A and B and D and F
>
> How many degrees of freedom have I used up?
> Is it 4 because there are 4 different sets, or is it 4*2*2 = 16?
> Whatever the answer, if my Sell rule is the reverse of buy rule
> would I then use up double that? Suppose condition A is the result
> of a 20 step optimization how does that change the numbers of df
> used up?
>
> Thank you in advance for insights and recommendations on good
books
> on the topic.
> Sursod
>
------------------------ Yahoo! Groups Sponsor --------------------~-->
Try Online Currency Trading with GFT. Free 50K Demo. Trade
24 Hours. Commission-Free.
http://us.click.yahoo.com/RvFikB/9M2KAA/U1CZAA/GHeqlB/TM
--------------------------------------------------------------------~->
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|