|
PureBytes Links
Trading Reference Links
|
Hi All,
Although I have never personally worked with any of the systems
covered by Futures Truth, I have no doubt that they are all curve-
fitted. Any 'system' that purports to specialize in one market is
optimized for that particular set of data.
Some people will say that different markets have individual
characteristics or personalities. This may be true to a limited
extent. However, in testing, a computer doesn't 'know' what market it
is examining. All the computer knows is a bunch of numbers (highs,
lows, closes), from which it attempts to produce an algorithm to
explain or predict price behavior.
For a system to be valid, it should work over a given set of numbers
(data). Whether those numbers have a name such as 'Beans" or 'Bonds"
is (and should be) irrelevant to the data and to the testing program.
Lots of systems make money when they trend and lose money when they
don't. This is not surprising. The best that you can hope for is to
create a system that is profitable over time over a wide range of
markets. Systems such as the Turtles use, makes money when the
markets trend and loses money when they don't (no surprise).
Since trendiness is a proven characteristic of commodity markets,
given a long enough sample period (i.e. ten years) almost all the
markets yield positive results.
However, in any given year, since there are only a few good trends,
most of the markets will prove unprofitable. This is not a reason to
abandon the system, or to eliminate (temporarily) unprofitable
markets from the portfolio. In fact, the markets that have lost the
most money recently (due to being in a consolidation) will probably
be the best in the future (when they finally hit a trend).
Finally, If one could learn to tell when the markets will trend and
when they will be in a trading range, they wouldn't need to know much
else to make money. One can visually eyeball a chart and tell if it's
in a trend or consolidation, but that still doesn't tell one much
about the future.
Regards,
Pal
--- In amibroker@xxxxxxxxxxxxxxx, "palsanand" <palsanand@xxxx> wrote:
> Steve,
>
> I don't use this Aberration system, so I can't comment on it. I do
> know that Futures Truth tests a lot of systems which are
essentially
> curve-fitted...
>
> I compare all systems with my primary trading system which works
for
> all markets (Stocks, Futures and FOREX) and has been in R&D for
over
> 45 years. It is also the most accurate T-Bond trading system ever
> developed... I have collected/developed over 140 systems but I'm
yet
> to find another system other than my primary trading system with
> which I can trade with absolute confidence and even convert my
losing
> positions into profit... The rest of the 139 or so systems have
been
> reduced to simple verification systems....
>
> George Patton once said that a Warriors greatest asset is his self-
> confidence...
>
> Regards,
>
> Pal
> --- In amibroker@xxxxxxxxxxxxxxx, "CedarCreekTrading"
<kernish@xxxx>
> wrote:
> > Pal,
> >
> > For a system to be valid, it must work on all numbers tested, not
> just those with certain names and not others
> > with different names.
> >
> > Try telling Keith Fitchen that (the author of Aberration, the
most
> successful mechanical system ever sold ... check out Futures
Truth).
> First, Keith will tell you that his wildly successful approach DOES
> NOT work on equities. Second, Keith will tell you that he only
> trades a basket of six commodities. I believe both these
principles
> are directly contrary to your opinions about optimizing and the
> selection of issues to be traded.
> >
> > If a system works on Bonds and not on Beans, this system is curve
> fitted over a specific set of data (Bonds) and it loses all
> statistical validity.
> >
> > Wrong, wrong, wrong. If I have an approach that has worked on
> Bonds for ten years and it doesn't work on beans...BFD. Should I
> abandon a robust approach to trading Bonds...because I can't
> make "beans" on Beans?
> >
> > Beware of drinking other people's bath water and whatever you do,
> don't drink the Kool Aid .
> >
> > Take care,
> >
> > Steve
> >
> >
> > ----- Original Message -----
> > From: palsanand
> > To: amibroker@xxxxxxxxxxxxxxx
> > Sent: Monday, October 20, 2003 9:43 AM
> > Subject: Objective functions (was RE: [amibroker] Re:
> Optimization -- again)
> >
> >
> > Hi,
> >
> > In my mind, curve fitting means either using different systems
> for
> > different markets, or using different parameters of the same
> system
> > for different markets, and this is not valid technical analysis.
> >
> > Historical testing via computer means feeding a set of numbers
> (open,
> > ow, close prices), and receiving back an output set of rules
that
> > hopefully will make money trading. The numbers themselves do
not
> have
> > names, and the computer doesn't recognize the difference
> > between 'Beans' or 'Bonds'. For a system to be valid, it must
> work on
> > all numbers tested, not just those with certain names and not
> others
> > with different names.
> >
> > If a system works on Bonds and not on Beans, this system is
curve
> > fitted over a specific set of data (Bonds) and it loses all
> > statistical validity. To believe it will work in the future as
it
> has
> > worked in the past is very dangerous.
> >
> > Also, different markets do not have different personalities.
> Again,
> > they are reduced to just being a set of numbers or a bunch of
> > algorithms. If a channel breakout (or any other) method is
> > successful, then the same parameter must be used for all the
> markets,
> > for the same reasons as above. You cannot use a 20-day channel
in
> > Silver and a 40-day channel in Corn, this also falls under the
> crime
> > of curve fitting.
> >
> > I therefore take exception to any system, that either only
trades
> one
> > specific market or group of markets, or trades different
markets
> > using different parameters or rules of the same system. All
this
> > proves is what has worked best in the past, and this will
usually
> not
> > continue to work in the future, as there is no correlation
under
> this
> > scenario.
> >
> > This is not specifically written to condemn vendors. This is a
> > clarification of my definitions of 'optimizing' and 'curve
> fitting',
> > and a warning as to what types of trading systems may be valid
> and
> > what to stay away from.
> >
> > Regards,
> >
> > Pal
> > --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill"
<dmerrill@xxxx>
> > wrote:
> > > thanks, I'll check it out if I can find it.
> > >
> > > I'm sure I'm ignorant, but how logic or sound trading
> principles
> > can be used
> > > to set an MA period (for instance) without examination of
past
> > history
> > > escapes me. as does the distinction between using past
history
> or
> > > 'experience' to do that and optimization. as does the
> justification
> > for
> > > seeing optimizations from one point in time as somehow
blessed
> > above all
> > > others.
> > >
> > > dave
> > > I would have to refer you to an article published by
Futures
> > Magazine
> > > concerning optimization and its research value in November
> 20?? by
> > > Kent Calhoun.
> > >
> > > Possibly the only way to do it correctly, is to first
arrive
> at a
> > set
> > > of parameters and algorithm based on logic, experience, or
> sound
> > > trading principals that won't be subject to change. Then do
a
> walk
> > > forward with no attempt to improve results via optimization.
> > >
> > > Regards,
> > >
> > > Pal
> > >
> > >
> > > --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill"
> <dmerrill@xxxx>
> > > wrote:
> > > > Pal, couple questions/comments.
> > > >
> > > > - are you saying that 30 "occurrences" in any system
> produces
> > 95%
> > > accuracy?
> > > > 30 trades? regardless of the market or trading system
rules
> or
> > time
> > > frame?
> > > > what's the basis for saying this?
> > > >
> > > > - could you explain "select stable parameters with an
> equity
> > shift
> > > less than
> > > > the parameter shift after equity spikes have been
> eliminated"? I
> > > don't
> > > > understand what you mean.
> > > >
> > > > - just fyi, your last paragraph seems to be trying to
> convince
> > me
> > > that
> > > > optimizing is good, probably in response to my asking "if
> you
> > > prefer not to
> > > > optimize parameters, how do you set them?". I asked that
> only
> > > because you
> > > > said, "I prefer a system to work without optimization",
> which I
> > > thought was
> > > > a nice goal, but one I don't understand how to achieve.
> seems
> > that
> > > you don't
> > > > actually intend to avoid optimization either, since you
then
> > > discuss how you
> > > > do it.
> > > >
> > > > dave
> > > > There is a correct method to optimize any system that is
> > > > statistically valid, 30 occurrences with 95% accuracy.
> > > >
> > > > The key to optimization is to select stable parameters
> with an
> > > equity
> > > > shift less than the parameter shift after equity spikes
> have
> > been
> > > > eliminated. This process creates stability for optimal
> > parameters
> > > > shifts within the four technical market phases.
Parameter
> > shift is
> > > > always geometric, but equity shift decline relative to
> > unstable
> > > > parameter selection is usually exponential.
> > > >
> > > > All systems are optimized to some degree. As soon as a
> trader
> > > chooses
> > > > to enter a trade on the open as opposed to the
> high/low/close
> > of
> > > day,
> > > > he has made a decision as to how a system should be
> traded.
> > Does
> > > he
> > > > know the high/low/close of day entry is better than the
> next
> > > opening
> > > > for an entry? If not why not? A potential 28%
difference
> in
> > > > profitability exists for channel system entries between
> opens
> > and
> > > > closes.
> > > >
> > > > The purpose of trading is to consistently make money.
> This is
> > > done by
> > > > having the best information available. If a trader does
> not
> > know
> > > the
> > > > best entry for his system, what is he trying to prove?
> That
> > the
> > > > system isn't optimized? To lose money because a trader
is
> > > ignorant of
> > > > his systen's best parameters is foolish.
> > > >
> > > > Regards,
> > > >
> > > > Pal
> > > > --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill"
> > <dmerrill@xxxx>
> > > > wrote:
> > > > > one question pal: if you prefer not to optimize
> parameters,
> > how
> > > do
> > > > you set
> > > > > them? or do you have some kind of trading rules that
> don't
> > have
> > > time
> > > > > constants, trigger levels, etc, that need to be set?
> > > > >
> > > > > dave
> > > > >
> > > > > I thought I might throw in my 2 cents.
> > > > >
> > > > > Vendors love optimization, because it can generate
> eye
> > popping
> > > > > hypothetical profits which has no connection to
real-
> time
> > > trading.
> > > > >
> > > > > I prefer a system to work without optimization. But
> if I
> > have
> > > to
> > > > do
> > > > > it, I would make sure that the optimization is
robust
> in
> > the
> > > > > following manner:
> > > > >
> > > > > 1. The sample size of data should be large enough
to
> > represent
> > > > real-
> > > > > time market conditions - bull, bear and sideways
> markets.
> > > > >
> > > > > 2. The look-back period should be as large as
> possible
> > for the
> > > > same
> > > > > reasons.
> > > > >
> > > > > 3. The testing of optimizable parameters should be
on
> out
> > of
> > > > sample
> > > > > data using walk-forward analysis.
> > > > >
> > > > > 4. The Central Limit Theorem says that for a sample
> to
> > assume
> > > the
> > > > > characteristics of the population, the size of
sample
> > should
> > > be
> > > > > large. The minimum sample size should be around 30.
> But
> > since
> > > an
> > > > > uptrend or downtrend can last for say 50 periods, I
> would
> > > have a
> > > > > minimum sample size of 100 periods making sure that
> the
> > full
> > > > market
> > > > > cycle is there (uptrend, downtrend and congestion).
> > > > >
> > > > > 5. The optimizable parameters should be as few as
> > possible and
> > > > tested
> > > > > in a wide variety of markets.
> > > > >
> > > > > Curve-fitting is like rolling a fair dice with 1/6
> > > probability of
> > > > > getting any number from 1 to 6, rolling it 5 times,
> > getting
> > > #6, 4
> > > > out
> > > > > of 5 times (80%) of time.
> > > > >
> > > > > A lot of traders fall in the trap of curve-fitting
> without
> > > being
> > > > > aware of it. So when designing a system, it is
> important
> > to
> > > keep
> > > > your
> > > > > guard up as far as curve-fitting is concerned.
> > > > >
> > > > > Regards,
> > > > >
> > > > > Pal
> > > > > --- In amibroker@xxxxxxxxxxxxxxx, "Gary A.
> Serkhoshian"
> > > > > <serkhoshian777@xxxx> wrote:
> > > > > > Fred,
> > > > > >
> > > > > > Could you narrow-down your idea of a reasonable
> sample
> > size
> > > for
> > > > > backtests. You've been hinting at rather sizeable
> > backtesting
> > > > > periods, but would like to put some numbers to it.
> Also
> > > wonder if
> > > > > you use # of trades as a guide versus period of
time
> for
> > > > backtesting
> > > > > period.
> > > > > >
> > > > > > Thanks,
> > > > > > Gary
> > > > > >
> > > > > > Fred <fctonetti@xxxx> wrote:
> > > > > > There are a lot of questions and provacative
> statements
> > in
> > > your
> > > > > post,
> > > > > > only one of which from my perspective needs an
> > > answer/response.
> > > > > >
> > > > > > Market behavior will continually change after
> that ...
> > > > > >
> > > > > > Change ? from what ? into what ? I guess this is
> the
> > part I
> > > > don't
> > > > > > follow. To me there is nothing new in market
> behavior
> > now
> > > that
> > > > > > didn't exist last month, last year, last decade,
> last
> > > century,
> > > > but
> > > > > > clearly those that take a short sighted view of
> history
> > and
> > > the
> > > > > > market action that made up that history will
> clearly
> > never
> > > see
> > > > it.
> > > > > > It's a forest and trees thing ...
> > > > > >
> > > > > > --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill"
> > > <dmerrill@xxxx>
> > > > > > wrote:
> > > > > > > I'm not trying to be argumentative, honest (:-
> )... I'm
> > > more
> > > > than
> > > > > a
> > > > > > little
> > > > > > > sick of saying the same thing over and over,
but
> I j
> > u s
> > > t
> > > > d o
> > > > > > n ' t g
> > > > > > > e t i t .
> > > > > > >
> > > > > > > ------------------------------
> > > > > > >
> > > > > > > I fail to see the huge difference in principle
> between
> > > equity
> > > > > > feedback and
> > > > > > > backtesting.
> > > > > > >
> > > > > > > let's start by assuming that backtesting
> performance
> > of a
> > > > system
> > > > > > and its
> > > > > > > parameters over some period of past data tells
you
> > > something
> > > > > about
> > > > > > its
> > > > > > > future performance. it's not a perfect
predictor,
> but
> > > it's the
> > > > > best
> > > > > > evidence
> > > > > > > we have. does this seem like a reasonable
> starting
> > point?
> > > what
> > > > > > alternative
> > > > > > > is there?
> > > > > > >
> > > > > > > if that's true, why is it better to do it only
> once?
> > what
> > > > > > justification is
> > > > > > > there for picking one examination period over
> another?
> > > clearly
> > > > > > market
> > > > > > > behavior will change continually after that.
> don't we
> > > need a
> > > > way
> > > > > of
> > > > > > working
> > > > > > > that looks at what's been happening and evolves
> our
> > > response?
> > > > > > >
> > > > > > > sounds like we examine performance up to some
> point
> > and
> > > > adjust,
> > > > > > trade with
> > > > > > > the best-choice system and parameters for a
> while,
> > then
> > > > examine
> > > > > and
> > > > > > adjust
> > > > > > > again later. make sense? what alternative is
> there?
> > > > > > >
> > > > > > > so then, how often do we re-examine performance
> > history?
> > > to
> > > > put it
> > > > > > > differently, how long do we ignore any changes
in
> > market
> > > > dynamics
> > > > > > that may
> > > > > > > or may not have occurred? why would
intermittently
> > > refusing to
> > > > > look
> > > > > > and
> > > > > > > respond improve system performance or
reliability?
> > > > > > >
> > > > > > > if that needs to be done, why not have the
system
> > itself
> > > do
> > > > it,
> > > > > as
> > > > > > part of
> > > > > > > its inherent operation? why is it better for us
> as an
> > > outside
> > > > > agent
> > > > > > to
> > > > > > > periodically run some separate tests, reach
into
> the
> > > > internals of
> > > > > > the
> > > > > > > system, and change stuff?
> > > > > > >
> > > > > > > or should we just continue with the system and
> > parameters
> > > we
> > > > > choose
> > > > > > at the
> > > > > > > beginning? are they somehow more valid than
what
> we'd
> > > choose
> > > > > later,
> > > > > > using
> > > > > > > the same backtesting methods, but on a
different
> date
> > > range of
> > > > > data?
> > > > > > >
> > > > > > > ------------------------------
> > > > > > >
> > > > > > > I realize that even if it seems to make sense
> > logically,
> > > this
> > > > all
> > > > > a
> > > > > > complete
> > > > > > > crock if no systems put together like this even
> > backtest
> > > well,
> > > > > > never mind
> > > > > > > forward testing.
> > > > > > >
> > > > > > > but every time I think about abandoning this
line
> of
> > > > research, it
> > > > > > seems like
> > > > > > > the first thing I'd want to do with a new
system
> > would be
> > > > (let me
> > > > > > guess),
> > > > > > > test and possibly adjust it using data up to
some
> > date,
> > > then
> > > > run
> > > > > > with it for
> > > > > > > a while after that and see if equity growth is
> good.
> > if
> > > it is,
> > > > > I'd
> > > > > > want to
> > > > > > > lather, rinse and repeat with other in and out
of
> > sample
> > > > data, to
> > > > > > make sure
> > > > > > > that wasn't coincidence.
> > > > > > >
> > > > > > > sounds way too familiar to be a completely
> different
> > > animal.
> > > > > > >
> > > > > > > dave
> > > > > > > From: Fred [mailto:fctonetti@x...]
> > > > > > >
> > > > > > > That IS what I was trying to say. I suspect
> because
> > > equity
> > > > > feed
> > > > > > back
> > > > > > > is like looking in a rear view mirror, great
for
> > > letting us
> > > > know
> > > > > > > where we were and how we could have adjusted
> the
> > past to
> > > > make it
> > > > > > > better, but that's about it.
> > >
> > >
> > > Yahoo! Groups Sponsor
> > >
> > >
> > >
> > > Send BUG REPORTS to bugs@xxxx
> > > Send SUGGESTIONS to suggest@xxxx
> > > -----------------------------------------
> > > Post AmiQuote-related messages ONLY to:
> amiquote@xxxxxxxxxxxxxxx
> > > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > > --------------------------------------------
> > > Check group FAQ at:
> > > http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> > >
> > > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> > Service.
> >
> >
> > Yahoo! Groups Sponsor
> > ADVERTISEMENT
> >
> >
> >
> >
> > Send BUG REPORTS to bugs@xxxx
> > Send SUGGESTIONS to suggest@xxxx
> > -----------------------------------------
> > Post AmiQuote-related messages ONLY to:
amiquote@xxxxxxxxxxxxxxx
> > (Web page: http://groups.yahoo.com/group/amiquote/messages/)
> > --------------------------------------------
> > Check group FAQ at:
> http://groups.yahoo.com/group/amibroker/files/groupfaq.html
> >
> > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> Service.
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Rent DVDs Online - Over 14,500 titles.
No Late Fees & Free Shipping.
Try Netflix for FREE!
http://us.click.yahoo.com/Tq9otC/XP.FAA/3jkFAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|