|
PureBytes Links
Trading Reference Links
|
>For what is worth, FFT is most often applied to audio signals, that
>are definitelly NOT stationary
Good point.
Stationarity is another term brought over from academia.
I don't have a good understanding of the concept.
I'm going to dig in on that one.
Thanks.
brian_z
--- In amibroker@xxxxxxxxxxxxxxx, "Tomasz Janeczko" <groups@xxx>
wrote:
>
> > Some of the 'academic' stats methods are not quite as good a fit
to
> > trading as we first assume e.g. look at the recent discussion on
> > Fourier Transform, which comes from Signal Processing
(electronics?)
> > and falls down rather spectacularly in trading because
stockmarket
> > data isn't stationary.
>
> For what is worth, FFT is most often applied to audio signals, that
> are definitelly NOT stationary :-) so that argument is not really
valid :-)
> There are methods (windowing+padding) to increase FFT resolution on
short data samples,
> and there are also methods different than FFT for spectal analysis.
> But that is rather broad subject ....
>
> Best regards,
> Tomasz Janeczko
> amibroker.com
> ----- Original Message -----
> From: "brian_z111" <brian_z111@xxx>
> To: <amibroker@xxxxxxxxxxxxxxx>
> Sent: Wednesday, March 12, 2008 12:41 AM
> Subject: [amibroker] Re: Statistical tests as custom metrics
>
>
> > Thomas,
> >
> > I have included Jeffs question with your opening query since they
fit
> > so closely.
> >
> >> looking at your algorithm I'm not seeing expected. What exactly
> >>are
> >> you test significance against?
> >
> > The expected value of an unbiased binomial event is 0.50 (50%
win).
> >
> > Since the (equity) market has a slight upward bias we need to use
an
> > expected value, that includes the market bias, as the base line
(or
> > detrend as per Howards/Aronsons methods).
> >
> > (Personally I don't detrend as I like to condition myself to see
the
> > markets as they actually are, and mentally factor in the trend,
so I
> > use market bias benchmarking).
> >
> >> > in "Quantitative Trading Systems" on p. 256, Howard describes
a z-
> >> score
> >> > test in order to evaluate the statistical significance of a
> > trading
> >> > system. While the formula is easy to write in AFL, I don't
think
> >> that
> >> > it can be done as a custom metric since the system to be
> > evaluated
> >> is
> >> > compared with a Random System. Any idea how to sensibly
implement
> >> it in
> >> > Amibroker?
> >
> > At page 91,in his book, (Entries and Exits chapter) Howard gives
some
> > very good (random entry) examples of how we can get an estimation
for
> > the 'standardized binomial expectancy' of any market i.e. you can
get
> > the mean expected wins for the actual market you are testing your
> > system in and use that in the Z score calculation - I think you
would
> > be better to use random entries with an exit after a set number
of
> > days == the average time your system trades are in the market.
> >
> > I am still learning AB myself so I am not sure if we can
implement
> > Howards Z equation directly in AB - you will probably have to do
it
> > outside somewhere - I haven't figured out how we can get the SD
of a
> > trade series (from a backtest) in AB - anyway we don't have built
in
> > stats tables (I suppose you could manually plug in the typical Z
> > scores).
> >
> > I'm exporting to Excel and doing my evaluations there, but I
don't
> > get that fancy.
> >
> > > > I'm using another statistical test proposed by the late
Arthur
> >> Merrill
> >> > some years ago in S&C. It's the "chi squared with one degree
of
> >> > freedom, with the Yates correction". Here's how I implemented
it
> > in
> >> AB:
> >> >
> >> > //chi squared with one degree of freedom, with the Yates
> > correction
> >> > wi=st.GetValue("WinnersQty");
> >> > Lo=st.GetValue("LosersQty");
> >> > Chi = (abs(wi-Lo)-1)^2/(wi+Lo);
> >> > bo.AddCustomMetric( "Chi-Squared modif.: >10.83: very
> >> > significant(1000:1), >6.64: significant (100:1) , >3.84:
probably
> >> > significant (20:1), <3.84: significance doubtful", Chi );
> >> >
> >> > What do you think about this metric?
> >
> > I think it is a very conservative measure.
> >
> > One of the problems we have, in evaluation, is that 'academic'
> > statistics filtered into freelance trading via institutional
> > investing - nothing against academics or institutional traders
but
> > their focus is somewhat different to freelance traders.
> >
> > Some of the 'academic' stats methods are not quite as good a fit
to
> > trading as we first assume e.g. look at the recent discussion on
> > Fourier Transform, which comes from Signal Processing
(electronics?)
> > and falls down rather spectacularly in trading because
stockmarket
> > data isn't stationary.
> >
> > Most of the stats we are using assume stationarity and also
assume
> > that data will be normal/ random i.e. it will have a normal
> > distribution and that the datapoints are independent of each
other.
> >
> > Neither is absolutely true, so the stats we are using are
> > approximations (of course the data we are using is only an
> > approximation anyway) - hence the doubts about Merrill's Chi.
> >
> >> > While this metric doesn't tell you anything if your system is
> >> > profitable, it tells you if its signals are only pure
coincidence
> >> > (simply put). It's remarkable that many systems that seem to
be
> >> > promising according to the usual metrics, are below 3.84, i.e.
> >> > significance doubtful. You need either a rather high number of
> >> trades
> >> > or a very high percentage of winning trades to shift this
metric
> >> > significantly higher. At least for (medium-term) EOD systems
> >> (that's
> >> > what I trade) this is not easy to achieve.
> >> >
> >
> > Yes, it is very hard to find good trading systems.
> >
> > This is what I have found - many tests that come up with nothing,
> > especially in the first two years.
> >
> >>>Are there other "better"
> >> > statistical metrics? If yes - would you mind sharing the AFL
code?
> >> >
> >
> > Try Howards Z method, using his random code, to find your
expected
> > win rate for your market and see how that works out.
> >
> > I have started some original (to me) work, based on binomial
> > simulation of equity curves and the behaviour of random, 50/50,
> > trading systems.
> >
> > It is only at the experimental, concept stage.
> > I intend posting it to the UKB one day so that the mathematically
> > trained people in the forum can critique it (it might be a load
of
> > old rubbish for all I know).
> >
> > Based on that work I am using PowerFactor, with sample error, to
> > guestimate significance (I can quickly do that in my head).
> >
> > Note that in PowerFactor the binomial component is considered to
be
> > Gaussian, with independent variables, while the distribution of
the
> > trades (ave%won/ave%lost) is not.
> >
> > Because of that I only apply the significance test to the W/L
> > binomial component (I claim that carrying out stats analysis on
the
> > compound system results is biased because of the non-normal
nature of
> > the distribution etc).
> >
> > For binomial events:
> >
> > variance == sample error (sort of)
> >
> > For 100 trades:
> >
> > sample error = +_10%;
> > expected random result (benchmark) == 50 wins;
> >
> > a no win trade will have a range of:
> >
> > 45-50 wins (one standard dev)
> > 40-60 wins (two standard devs)
> > 35- 65 (three standard devs) etc
> >
> > So for 100 trades 60 wins doesn't happen all that often, if the
coin
> > is a fair coin (a random event).
> >
> > We are defintely going to take notice of 60/100 wins BUT it is
> > not 'out of this world' and we do not have certainty - we only
have
> > the expectation that it is good - reality can, and does, dash
> > expectations on occasion.
> >
> > Because of this, W/L results are never a sure thing.
> >
> > To ensure against this take control of the ave%W/ave%L ratio -
that
> > is something we can control via stops - if the W/L ratio turns
out to
> > be a 'BlackSwan' our good stops will save us from crashing and
> > burning (keep us at low drawdowns).
> >
> > Comparing to Chi (for 100 trades with a 60% win record):
> >
> > Chi = (abs(wi-Lo)-1)^2/(wi+Lo);
> >
> > == ((60-40)-1)^2/(wi+Lo);
> > == 19^2/100;
> > == 361/100
> > == 3.6
> > == Not significant according to Chi but significant according to
> > brian (always look on the bright side of life!).
> >
> > I agree that finding 60% winners, in any market or timeframe, is
very
> > difficult - that is the reality of trading.
> >
> > This problem is especially prevalent in mid - long term trading -
say
> > indicators with long lookbacks are used - then the number of
signals
> > available tends towards becoming a rare event and the trader then
can
> > only see a small part of the longterm (10000 plus) trades - the
> > trader soon runs out of clean data and can't get high enough
trade
> > counts.
> >
> > That is why I like shorter term trading (intraday to 2-3 day
cycles)
> > where I can take advantage of statistical smoothing (I quickly
> > approach my theoretical edge i.e. relative to the calendar days).
> >
> >
> > As I said - please use 'my' theories at your own risk, at least
until
> > after I post on the topic, and the mathematicians in the forum
have a
> > chance to bash up my hypotheses.
> >
> >
> > brian_z
> >
> >
> >
> >
> >
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "jeffro861" <jeffro861@> wrote:
> >>
> >> Ok, so the chi-squared tests for independence (real vs.
expected)
> > so,
> >> looking at your algorithm I'm not seeing expected. What exactly
> > are
> >> you test significance against?
> >>
> >> --- In amibroker@xxxxxxxxxxxxxxx, Thomas Ludwig <Thomas.Ludwig@>
> >> wrote:
> >> >
> >> > Hello,
> >> >
> >> > in "Quantitative Trading Systems" on p. 256, Howard describes
a z-
> >> score
> >> > test in order to evaluate the statistical significance of a
> > trading
> >> > system. While the formula is easy to write in AFL, I don't
think
> >> that
> >> > it can be done as a custom metric since the system to be
> > evaluated
> >> is
> >> > compared with a Random System. Any idea how to sensibly
implement
> >> it in
> >> > Amibroker?
> >> >
> >> > I'm using another statistical test proposed by the late Arthur
> >> Merrill
> >> > some years ago in S&C. It's the "chi squared with one degree
of
> >> > freedom, with the Yates correction". Here's how I implemented
it
> > in
> >> AB:
> >> >
> >> > //chi squared with one degree of freedom, with the Yates
> > correction
> >> > wi=st.GetValue("WinnersQty");
> >> > Lo=st.GetValue("LosersQty");
> >> > Chi = (abs(wi-Lo)-1)^2/(wi+Lo);
> >> > bo.AddCustomMetric( "Chi-Squared modif.: >10.83: very
> >> > significant(1000:1), >6.64: significant (100:1) , >3.84:
probably
> >> > significant (20:1), <3.84: significance doubtful", Chi );
> >> >
> >> > While this metric doesn't tell you anything if your system is
> >> > profitable, it tells you if its signals are only pure
coincidence
> >> > (simply put). It's remarkable that many systems that seem to
be
> >> > promising according to the usual metrics, are below 3.84, i.e.
> >> > significance doubtful. You need either a rather high number of
> >> trades
> >> > or a very high percentage of winning trades to shift this
metric
> >> > significantly higher. At least for (medium-term) EOD systems
> >> (that's
> >> > what I trade) this is not easy to achieve.
> >> >
> >> > What do you think about this metric? Are there other "better"
> >> > statistical metrics? If yes - would you mind sharing the AFL
code?
> >> >
> >> > Best regards,
> >> >
> >> > Thomas
> >> >
> >>
> >
> >
> >
> >
> > Please note that this group is for discussion between users only.
> >
> > To get support from AmiBroker please send an e-mail directly to
> > SUPPORT {at} amibroker.com
> >
> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > http://www.amibroker.com/devlog/
> >
> > For other support material please check also:
> > http://www.amibroker.com/support.html
> >
> > Yahoo! Groups Links
> >
> >
> >
>
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|