|
PureBytes Links
Trading Reference Links
|
Dave,
<it's interesting that you propose "complicating the issue by adding
further pattern filters". most talk I've heard about systems that are
truly robust in the long-term says the opposite, go simple.>
Now that you mention it, I failed to either elaborate (or think about)
certain reasons that caused me to start tinkering with added pattern
filters in the first place, so here it is:
Before AB, I had my own backtester that I developed myself and did not
have nearly the sophistication that AB now offers.
I would develop trading systems against a 10 yr 10,000 issue db and
some of these systems were profitable, but generated *huge* numbers of
trades that were beyond my own ability to trade. So I started adding
the pattern filters and accomplished two things. First I reduced the
number of trades to the level that I could actually trade the
system(s), and second I discovered that the pattern filters could
double or triple the original RAR measurements and %profitable
numbers. So I had the best of both worlds.
But slippage has become an increasing trading issue of late, so I have
shifted my trading and backtesting to a basket of only 300 to 600
stocks that account for 66% to 80% of $ volume of all listed stocks on
the US exchanges in order to try to minimize slippage. This reduced
basket of stocks has now caused the number of setups from my systems
to decline significantly and I am just now starting to play with my
original pattern filters to see what I might be able to change in
order to increase my number of setups with a miminum negative impact
on backtest results.
But my point was still valid insofar as discovering that additional
pattern filters in some backtests did have positive impact on RAR and
other measurements.
The rest of your post was interesting. Sometimes I need to think about
things before deciding if I can contribute anything thru a response.
You've probably noticed that too often I post replies to things
without contributing much. :-(
Regards,
Phsst
--- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx> wrote:
> thanks for jumping in Phsst, and no, you haven't overstayed your
welcome,
> not by a long shot (:-). if I poke and prod in my return, it's
exploration,
> not disagreement.
>
> it's not exactly that I haven't developed any strategies that show a
profit.
> that I've done, repeatedly. it's that nothing I've done has given me the
> confidence that it was arrived at by a process that consistently
identifies
> profitable systems. that's really what auto-optimization is about,
defining
> an objective process for evaluating systems that predicts profitability
> reasonably well.
>
> to put it differently, if I test a system over one range of time and
it does
> well, do I know what I have and haven't proved? what does it mean if
someone
> says (not just here, btw) that they've backtested CCI(25) crossing
zero on
> some index since 1988 and it works well enough that it's possibly a good
> general market timing signal? suppose I wanted to trade a stock universe
> better represented by a different index, and wanted to investigate a
good
> period to use. I can optimize over all data to date, look for ranges of
> profitable settings, and pick one in the middle, as b and herman have
> suggested. but would I have gotten the same result if I'd done that
in 2000,
> or '95? if not, why should I buy any one of those results, or what that
> other person said, or the conventional wisdom? most importantly, why
should
> I have confidence in my process should I want to evaluate another
system?
>
> *that's* why I built the auto-optimizer originally, not to trade
constantly
> re-optimized systems, though of course I wondered how well that
would work
> too. but the failure of auto-optimization in its current form
confirmed my
> feeling that I didn't really know how to predict the profit a system
would
> make going forward. that means I shouldn't be taking my results in the
> system selection biz very seriously until I get my act together.
make sense?
>
>
> I don't think of myself as believing much of the "recommendations" I
hear
> without investigation. if I was easier about that, I'd be trading
this stuff
> sooner, instead of still testing and thinking (:-). also, I didn't
"assume"
> that the test date range mattered a lot, I found that it did
experimentally,
> at least the way I was testing. can you give me an example of a trading
> system or way of testing that's more stable over time? privately if you
> want, discretion assured.
>
> it's interesting that you propose "complicating the issue by adding
further
> pattern filters". most talk I've heard about systems that are truly
robust
> in the long-term says the opposite, go simple. more than once, I've
tried
> taking only trades recommended simultaneously by several moderately
> profitable indicators, and haven't been that impressed.
>
> anyway, how can you confirm that the 8500% return you backtest on a
> double-mojo-bear-star-winky-thing coincident with DEMA(29.5) of the VIX
> crossing 73.1 on alternate tuesdays when the moon is full isn't a
curve fit?
>
> dave
> <...the general vibe was that this strategy and settings were and had
> been somehow golden for forever. (of course I'm exaggerating; these
> are smart, unpretentious people, but the underlying logic still seemed
> like that to me.)>
>
> Dave,
>
> Before I start here I'll preface my comments with the disclaimer that
> I am relying on memory which is not a particularly reliable thing
> where I am concerned...
>
> You're a bright guy. But don't I recall that you posted as recently as
> two or three weeks ago that you have not yet developed a profitable
> mechanical trading system using AB? Hey, I could be wrong and if I am
> then I apologize.
>
> I should probably stop here and wait for a response, but time is not
> on any of our sides, so I'll press on <g>...
>
> I would speculate that your original premise was correct in that a few
> people here do in fact have strategies and settings that have been
> reliable over a *very* long period of time and that there is no
> exaggreation in that assumption.
>
> < but I kept hearing things like "CCI(something)", and "MA(this) cross
> MA(that)", with some single test and/or optimization period to back it
> up.>
>
> You cannot judge anything based upon what you think you have heard
> here. If nothing else, you can count on the fact that those who have
> good profitable trading systems here are NOT going to part with them
> easily.
>
> <I concluded that what you'd recommend depended to a huge degree on
> what dates you looked at. that's one of the main reasons why I wrote
> this, to automate optimization over a bunch of time periods, and see
> if things stayed constant and/or profitable. neither. sowhattheheck.?
>
> My advice is to quit taking the recommendations of others, and quit
> assuming that anything depends to a 'huge degree' upon what dates are
> looked at.
>
> I'll go even further in a positive direction:
>
> Look at conventional wisdom related to indicators, but NEVER STOP
> THERE... be sure to go the extra distance and complicate the issue by
> adding EXTRA pattern filters to your backtests.
>
> By combining conventional wisdom with pattern filtering, you might
> discover that there are in fact long term strategies that make
> predictable profits under known mkt conditions.
>
> As usual, I always take too long to get to my point...
>
> You have abandoned the basic search for 'root' strategies in favor of
> developing a complex 'auto optimization' piece of code that seeks to
> ferret profits out of improbable strategies.
>
> Here is my advice to you...
>
> Focus upon backtesting basic technical strategies combined with
> pattern matching. Then, only after you find promising trading
> opportunities should you seek to optimize the various aspects of the
> stradegy.
>
> I might be wrong, but I suspect that all of your effort related to
> this 'auto optimization' code is not going to yield you any results
> until you have done the more basic task of developing 'core'
> strategies that show promise.
>
> Again, as usual I've talked too much and worn out my welcome.
>
> Best regards,
>
> Phsst
>
>
> --- In amibroker@xxxxxxxxxxxxxxx, "Dave Merrill" <dmerrill@xxxx>
wrote:
> > Isn't there some aspect of past performance (results) that can
> "persist"
> > for some time. That is to say, that a stock or mutual fund that is
> > profitable for awhile (esp from a backtest or ranking perspective)
> does not
> > and probably will not remain profitable "forever". However, it can
> and does
> > remain a good performer for some period.
> >
> > you'd think so, wouldn't you? what's the point of backtesting
> anything ever,
> > if the future can't be at least kind of, sort of, approximately,
> > semi-fake-estimated from what happened in the past?
> >
> > but I kept hearing things like "CCI(something)", and "MA(this) cross
> > MA(that)", with some single test and/or optimization period to back
> it up.
> > people didn't usually talk about optimizing over one time
period, then
> > testing with those optimal settings for a while *after* that, then
> > optimizing again later, etc.. the general vibe was that this
> strategy and
> > settings were and had been somehow golden for forever. (of
course I'm
> > exaggerating; these are smart, unpretentious people, but the
underlying
> > logic still seemed like that to me.) when I optimized some of
those same
> > things myself, over different time periods, I concluded that
what you'd
> > recommend depended to a huge degree on what dates you looked at.
> that's one
> > of the main reasons why I wrote this, to automate optimization over
> a bunch
> > of time periods, and see if things stayed constant and/or
profitable.
> > neither.
> >
> > sowhattheheck.
> >
> > What parameter(s) measure this? It probably changes over time
but it
> > seems a combination of return along with absence of large volatility
> is a
> > key measure. The UPI or UI or even CAR/Mdd seems like it captures
> more than
> > just raw return. While this seems more true for mutual funds
(actively
> > managed ones at that), it can also be true for certain stocks in
certain
> > industries.
> >
> >
> >
> > Just rambling out loud..I have not had time to try your
> system----can it
> > optimize against CAR/Mdd? for example?
> >
> >
> >
> > Ken
> >
> > as shipped, the framework comes with two metrics to choose from,
> profit per
> > bar, and net bars on the right side of the market per bar, meaning
> basically
> > this:
> > ((bars long and rising + bars short and falling) - (bars long and
> falling
> > + bars short and rising)) / number of bars
> >
> > both metrics can penalize the score by a selectable percentage
of max
> > drawdown. for instance, at zero penalty, MDD doesn't effect the
> score; at
> > 50%, 20% MDD reduces the score by 10%; at 100%, by 20%.
> >
> > you can also add your own metrics, AND OBVIOUSLY YOU SHOULD, because
> neither
> > of these are that great, or none of the trading rules tested are, or
> > something. another one I've thought of is some kind of measure
of the
> > smoothness of the equity curve, which is MDD is a crude look at.
> many of the
> > standard metrics are something of a pain to code on the results
of the
> > Equity function. hopefully, at some point tomasz will give us
AB's full
> > suite of report metrics, as functions that work off Equity.if you
> haven't
> > check the framework out yet, I'd suggest waiting for the
> new-and-improved
> > v1.1, due out soon.
> >
> > dave
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|