|
PureBytes Links
Trading Reference Links
|
I find "degree of freedom" as applied to trading system development
difficult to grasp and use (Robert Pardo, "Design, Testing, and
Optimization of Trading Systems" ) .
Insights please!
Pardo wrote statisticians use the term degree of freedom (df) as a
measurement of confidence in the test reults. Fewer specifications
and more data give greater confidence but there is no formula that
expresses the relationship (written in 1992, does a formula exist
now?) Pardo made some suggestions:
Degree of freedom = Number of signals – Number of rules,
or
90% degree of freedom (factor of 10):
Minumum df = 10 x (Rules + Conditions)
He wrote a 30 day moving average should be tested on at least 300
days of data (factor of 10) though satisfying the minimum
requirements is not as good as using fewer rules and more data to
produce a large number of trades ( P.56). Further on (P.140) he
mentioned a system performing 980 tests (980 optimization
combinations) on 1250 data points does not give confidence. My
question - what factor to apply on the number of optimization tests
to give an adequate optimization period, a factor of 10 is difficult
to uphold since even simple systems run up a large niumber of tests.
Take a simple MACD eod system with buy and sell using the same
settings. If we optimize using 10 steps for the "slow", "fast"
and "sig" moving averages we have 10x10x10 = 1000 combinations even
without profit or loss stops. A factor of 10 would require 10,000
days or close to 40 years of history for this simple system. A
trading system development tutorial I found in the Amibroker support
zone website optimizes 7x9x21x11x21x50x50 steps (smoothing, CCI,
lookback, Blevel, SLevel, stop, stop) which gives 764,032,500 tests.
These examples make me doubt my understanding of the factor of 10
rule, but factor or no factor it is still true the more we optimize
the larger the test window.
Going back to the MACD example, suppose the 3 moving averages are
optimized from
12 to 21 in steps of 1 = 10 tests
18 to 27 in steps of 1 = 10 tests
4 to 12 in steps of 1 = 10 tests
Can we say they use up 30 degrees of freedom? Since we do not count
overlaps (based on an example Pardo gave)and there are 24 steps from
4 to 27 can we say this system uses 24 df? In this case would the
90% df rule require a test sample to yield approx. 240 to 300 trade
signals for statistical confidence? Whatever matrix we use the test
window should naturally also be large enough to cover enough market
conditions etc.
My next question – suppose my buy rule is:
Buy = (A and B and (C or D) and (E or F);
The possible permutations:
A and B and C and E
A and B and C and F
A and B and D and E
A and B and D and F
How many degrees of freedom have I used up?
Is it 4 because there are 4 different sets, or is it 4*2*2 = 16?
Whatever the answer, if my Sell rule is the reverse of buy rule
would I then use up double that? Suppose condition A is the result
of a 20 step optimization how does that change the numbers of df
used up?
Thank you in advance for insights and recommendations on good books
on the topic.
Sursod
------------------------ Yahoo! Groups Sponsor --------------------~-->
Try Online Currency Trading with GFT. Free 50K Demo. Trade
24 Hours. Commission-Free.
http://us.click.yahoo.com/RvFikB/9M2KAA/U1CZAA/GHeqlB/TM
--------------------------------------------------------------------~->
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|