|
PureBytes Links
Trading Reference Links
|
Chris,
No, we don't have to be Quants to trade.
I actually only use basic stats in a simplistic way - it is how and
where you use it that counts (I did have to dig into a lot
of 'theory' to find that out and find what I wanted though).
All of the authors I mentioned, except for Kennedy & Bernstein, are
trading orientated so pick one or two that you like and dig in (that
should take you up the easy 80% of the learning curve).
Some further discussion:
(I'll try to keep it short and give you some practical pointers -
which everyone knows is hard for me to do).
1) SAMPLE ERROR
Sample error is very real.
I have plotted thousands of pseudo stock charts using the random
generator in Excel and watched sample error play out time and time
again.
Plot sample error (on the Y axis) against the number (N) of data
samples (on the X axis) and it is an inverse exponential curve.
Sample error% = 1/sqrt(N) * 100
The reduction in error that we gain as we increase sample size
decreases exponentially.
There is an optimal point somewhere around N = 1000 where we don't
get much improvement in accuracy with increased sample size.
Having seen sample error in thousands of labtests I find it very
difficult to accept any analysis that uses less than 300-400 samples
(as a minimum). If I can get more I use them (in my newest system
that I am working on I am getting thousands of trade samples per
test).
I go so far as to use data from other markets and timeframes to
produce my large datasets (plus other tricks as well).
2) SAMPLE ERROR IN OPTIMIZATION
I am very sceptical about optimization that ignores sample size.
When you have a nice round optimization 3D graph how meaningful could
it possibly be if it is produced from only 30 trade data points. The
sample size behind the plots on that graph are not evenly distributed
and possibly not statistically valid.
I haven't found any trading commentator who discusses this point or
gives examples of how to account for sample error when optimizing.
I assume it could be included as part of the test criteria (objective
function)?
Perhaps people are doing things like that in private and just haven't
told anyone about it.
3) SYNCHRONIZING TO THE CURRENT MARKET
I am very sceptical about this (once again it places the trader in
the position of relying on very small datasets).
Returning to the analogy of vehicle performance tests.
Unlike F1 racing we don't know anything about the track or the
weather (they know everything about in advance and prepare their car
accordingly).
Since we can't predict what track we will be racing on or what
conditions we will be racing under why would we prepare as if we do?
I would rather use a system that performs reasonably well under all
conditions than try to tune-up to unpredictable and ever changing
conditions.
brian_z111
--- In amibroker@xxxxxxxxxxxxxxx, ChrisB <kris45mar@xxx> wrote:
>
> Thanks Brian, just the sort of feedback and references I was
looking for:
>
> I guess you have put this into English better than I did:
>
> 3) ongoing re-optimization of a system to 'synchronise' with current
> (short term) 'market conditions'.
>
> This is the crux of the question, as I have tried to verbalize in
my
> response to Howard above, but not as succinctly as you have.
>
> This then is the philosophical question: is it better
to "synchronize"
> frequently and often and stay in tune with market movements, or
better
> to perform endless walkforwards? I for one wouldn't care less if a
> system stopped working as long as I could detect that early enough
and
> then just stop trading it. After all, as Howard says in his book
(and in
> earlier posts to this board), all systems stop working after a
while. I
> want to be trading this before it stops working and not later. I
look
> forward to the rest of Howard's book (and then a number of re-
reads) and
> may post some more questions here.
>
> One tedious approach would be to place sample trades from a number
of
> systems on a demo account, collate the data, plot the expectancy
and
> other stats into Excel and then just trade whichever one is
currently
> performing best. My goal in understanding how the AB backtester
works
> and how to apply it practically to my chosen market, is to speed
this up
> and remove the tedium from this.
>
> Thanks for the names of the other authors: will need to look these
up
> but hope that they are not too "quantsy" for my intellectual level.
As
> you say we should be able to trade without needing PhDs. (If I am
wrong
> here please correct me asap: at my age I may prefer to engage in
other
> activities in my declining years rather than pursuing a futile
exercise.)
>
> Thanks for the UKB link and associated references. That amount of
> reading should keep me off the streets for awhile!
>
> Regards
>
> ChrisB
>
>
>
>
>
> brian_z111 wrote:
> >
> > Hello Chris,
> >
> > Yes, I have some thoughts on the subject and a reference or two.
> >
> > In fact I am probably part of the way towards a decent book on the
> > subject (presented from a traders point of view) although I doubt
> > that I will ever write it.
> >
> > You actually have three topics there:
> >
> > 1) statistics (statistical significance) ,
> > 2) statistical significance as it is applied to optimization
> > (selecting the 'best performer' from a number of possible
parameters),
> > 3) ongoing re-optimization of a system to 'synchronise' with
current
> > (short term) 'market conditions'.
> >
> > For effective consideration it would be better to separate the
three
> > subjects, as far as possible, although from a traders point of
view
> > they all come under the one heading of "system design, testing and
> > evaluation" )SDT&E.
> >
> > The best people for you to ask about that subject, in the context
of
> > your question, would be those who are making a living using
systems
> > that they designed, using optimization as a key part of the
process,
> > or specialists in the field.
> >
> > I'm not in either of those categories.
> >
> > I do, however, know a few who are. For a start we have our very
own
> > Fred and Howard, who fit into one, or both.
> >
> > Then, there is Pardo, who is the father of SDT&E as we know it
> > (outside of academic circles)as well as Prof David Aronson, Lars
> > Kestner, Stridsman, Taleb.
> >
> > None of them are bad reads; some are better than others.
> >
> > In fact Aronson is my No1 favorite trading book and Howards book
is
> > in my top five. Aronson has a strong intellect and couples that
with
> > the ability to make a difficult subject sound easy as well as
being
> > readable and entertaining.
> >
> > IMO one of the handicaps we are under is that the subject evolved
> > from, and has been dominated by, academia while what we actually
need
> > is a pragmatic amnual of "applied stats for traders". Aronson's
book
> > goes a fair way, albeit not all the way, towards achieving that.
> >
> > (As far as I am concerned, anyone who thoroughly understands
Aronsons
> > work, and can apply it, knows 95%, or more, of everything they
need
> > to know on the subject).
> >
> > Howards book is amongst my top 5 because it is training orientated
> > and full of examples written in my one and only programming
langauge
> > (AFL). It complements Aronson nicely.
> >
> > Pardo is a little old and IMO doesn't go far enough, although to
be
> > fair I feel he did intend it to be an introductory text.
> >
> > Stridsman isn't too bad at all.
> >
> > Klestner disappointed a little although his claim to fame is the K
> > Ratio.
> >
> > Taleb I found neither entertaining nor informative although others
> > seem to recommend it.
> >
> > There's a rudimentary post on statistics at the UKB - I will file
> > details of the authors I have mentioned above to that post in the
> > next day or two. It's easier for me to put them there which saves
me
> > having to repeat this post in the future. There are a few bits and
> > pieces there - nothing much really - I hope to add to it one day.
> >
> > If you search this forum under WhitneyBroach OR Whitney (author)
AND
> > Aronson (contents) you will find some discussion (Whitney first
> > posted on Aronson).
> >
> > I might post some more discussion later but in the meantime here
is
> > something to think about:
> >
> > Your doctor informs you that you have an incurable, painless,
disease
> > and that you only have two weeks to live (there's absolutely no
> > chance of a cure and you will die peacefully in your sleep).
> >
> > As a last resort he offers you an experimental pill that has been
> > designed to extend our longevity.
> >
> > The pill has been trialled on 100 people.
> > 53% were totally cured and lived healthily and happily until they
> > were 120 (they died peacefully in their sleep too).
> >
> > The other 47% also lived until they were 120- but they suffered
> > horrendous side effects and lived out their lifes in unimagineable
> > pain (the pill over-rides any attempts at euthenasia).
> >
> > Will you take the pill or not?
> >
> > For you statistics is no longer a theoretical subject.
> > Your decision will have very real, life changing consequences.
> >
> > Now it is a fact that the majority of people will choose to take
the
> > pill; it's in our makeup to do so (the pain is in the future and
> > not 'real' at the time of making the decision, we emphasis the
> > positive and eliminate the negative, 53/47 sounds like good odds
to
> > most of us etc).
> >
> > But, if we step back a pace and evaluate the problem objectively,
> > rationally, logically (would one of those have been enough?) what
> > decision would we make then?
> >
> > How about if we are a trained statistician, or maybe a
professional
> > trader?
> >
> > How about if the Doctor said' "You don't have to decide now. There
> > will be a new trial, using 10,000 people, starting tomorrow and
> > finishing in three days; the results will be known immediately.
You
> > can wait until then to decide if you like, but there is only a
> > limited number of pills available and there is no guarantee that
you
> > will still receive one in four days time"?
> >
> > What is your answer now?
> >
> > IMO it's a binomial problem.
> >
> > http://www.amibroke r.org/userkb/ ?s=Statistics
> > <http://www.amibroker.org/userkb/?s=Statistics>
> >
> > brian_z
> >
> > --- In amibroker@xxxxxxxxx ps.com
> > <mailto:amibroker%40yahoogroups.com>, ChrisB <kris45mar@ ..>
wrote:
> > >
> > > What is a valid or reasonable number of backtest results to
subject
> > to
> > > Optimization?
> > >
> > > For general statistics a minimum of 30 or so is needed to start
> > getting
> > > valid StdDevs etc.
> > >
> > > If I run a backtest on hourly currency data over three months I
get
> > > around 16 -20 tradeable signals per currency.
> > > This give a nice smooth plateau on 3D optimization.
> > >
> > > If I test over two months of data I get around 10 - 12 trades
> > >
> > > If I test over only 1 month I get only 5 or 6 trades.
> > >
> > > These shorter time periods still give visually acceptable 3D
> > plateaus
> > > but I am wondering if there is enough data to be statistically
> > significant.
> > >
> > > I am trying to get a handle on how close I can get to current
> > > fluctuations in the market without hitting noise. The idea
being to
> > redo
> > > the Optimization every x time frame and shift the entry and exit
> > > parameters to stay in the middle of the plateau.
> > >
> > > Of course I can backtest over longer time frames, say 6 months
of
> > data,
> > > shifting the starting date forward by one month at a time, but
this
> > > would seem to introduce more "lag" into my selection of best
> > parameters
> > > to trade.
> > >
> > > Does anyone have any thoughts/references on this?
> > >
> > > --
> > > Regards
> > >
> > > ChrisB
> > >
> >
> >
>
>
> --
> Regards
>
> ChrisB
>
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|