|
PureBytes Links
Trading Reference Links
|
<BLOCKQUOTE
>
----- Original Message -----
<DIV
>From:
Richard
Alford
To: <A title=amibroker@xxxxxxxxxx
href="">amibroker@xxxxxxxxxxxxxxx
Sent: Thursday, June 20, 2002 2:17
PM
Subject: Re: [amibroker] Trends, random
series
I have an understanding of out of sample testing; that was
an issue.
However, doesn't the design range depend on whattype
of system, environment and universe one is considering? I doubt that the
90's have a lot to do with the present, at least for some time yet. Is
this an a priori assessment? Yes.
This is what I meant by"ensure
that the period used for development includes a variety of market conditions,"
unless one wants a bull/bear/flat system.
The recent discussions of an example CMO system that,when
tested on the NDX gave consistent from the 90's and also since 2000 was an
example. Using the NDX was certainly biased towards techs. The
only survivor of the system over both periods from my
calculations and recollection was ALTR.
There may be no universal solution only local
solutions.
Cheers,
Richard
<BLOCKQUOTE
>
----- Original Message -----
<DIV
>From:
wavemechanic
To: <A title=amibroker@xxxxxxxxxxxx
href="">amibroker@xxxxxxxxxxxxxxx
Sent: Thursday, June 20, 2002 12:41
PM
Subject: Re: [amibroker] Trends,random
series
<BLOCKQUOTE
>
----- Original Message -----
<DIV
>From:
<A title=richard.alford@xxxx
href="">Richard Alford
To: <A
title=amibroker@xxxxxxxxxxxxxxx
href="">amibroker@xxxxxxxxxxxxxxx
Sent: Thursday, June 20, 2002 1:26
PM
Subject: Re: [amibroker] Trends,
random series
Why wouldn't one simply design on now - 6 months,and
confirm on the previous 6 months? The suggestion of designing to
today, then testing for the next 6 months doesn't appear to be any
different - if one is absolutely certain they did not use
information from the preceding 6 months... Of course, selecting
systems that perform well on the last 6 out of sample data does use that
data - tis a puzzlement. Certainly only trying a new system every 6
months would constrain losses.
Using a portion of existing data
for system development and the balance for out of data testing is the
standard procedure, thereby having the ultimate Scrambler. However,
one of the pitfalls in evaluations of this nature is being able to ensure
that the period used for development includes a variety of market
conditions, which usually requires a fair amount of data. Otherwise,
one probably only has a system that is good for bull/bear/flat markets,
etc.
wrt Al's point - I will let him manage that one. I
am marketing bullet proof boots, btw.
Cheers,
Richard
<BLOCKQUOTE
>
----- Original Message -----
<DIV
>From:
Ken Close
To: <A
title=amibroker@xxxxxxxxxxxxxxx
href="">amibroker@xxxxxxxxxxxxxxx
Sent: Thursday, June 20, 2002 11:35
AM
Subject: RE: [amibroker] Trends,
random series
<SPAN
class=170473116-20062002>William:
<SPAN
class=170473116-20062002>
I think
the point Al was making with his idea of using the Simulator is that you
save TIME. If you optimize (bad word--deduce a system) based on
the last two years up until yesterday, and then you wish to test
"out-of-sample" for 6 months, you will have to sit and collect the next
six months worth of data before you can draw a conclusion about the
worth of the system you have developed. The idea behind the
simulator as I understand it is to generate the next six months worth of
data NOW and if results are satisfactory, THEN begin trading
tomorrow.
<SPAN
class=170473116-20062002>
The
purpose is to save time.
<SPAN
class=170473116-20062002>
However,
you shoot yourself in the foot if the conclusion is that the simulated
data is worthless as an out-of-sample test.
<SPAN
class=170473116-20062002>
I think I
have Al's purpose stated correctly.
<SPAN
class=170473116-20062002>
Ken
<FONT face=Tahoma
size=2>-----Original Message-----From: wpeters_1
[mailto:wpeters_1@xxxx]Sent: Thursday, June 20, 2002
1:17 PMTo: amibroker@xxxxxxxxxxxxxxxSubject: RE:
[amibroker] Trends, random series
<FONT face=Arial color=#0000ff
size=2>Thank you Al for that..i'm not sure that you
saw this follow up of mine:
<FONT face=Arial color=#0000ff
size=2>
<FONT
face=Arial><SPAN
class=875075116-20062002>"You then have to go on and ask what
would be the value of studying biases introduced in a series of
randomised data by mathematicians which does not occur in the real
world market place.
<FONT
face=Arial color=#0000ff size=2>
<FONT
face=Arial>Surely what wouldbe
valuable is to study and root out biases already in the marketdue
to human behaviour or whatever. And get to learn about human behaviour
and the effect on the market place.<SPAN
class=875075116-20062002>"
<FONT
face=Arial><SPAN
class=875075116-20062002>
<FONT
face=Arial><SPAN
class=875075116-20062002>
<FONT
face=Arial><SPAN
class=875075116-20062002>
<FONT
face=Arial><SPAN
class=875075116-20062002>Also-
<FONT
face=Arial>I
still don't see the value in this approach over using real
data history. If at the core of your argument you are saying there is
some possibility of this system reflecting the future, what might
happen in the future etc, then so could the data from the
past..
<FONT
face=Arial><SPAN
class=875075116-20062002>
<FONT
face=Arial>For
me it always comes back to 'where is the value' over straight ticker
history, which we know is as real as its going to get. Humanistic
patterns have been more or less the same over many years at a base
level and they are represented in history of the chart for us
to study. Any tinkering with those patterns and you get some artificial
result which will never happen in the real world market....and again
what is the worth of studying that data over the
real?
<FONT
face=Arial><SPAN
class=875075116-20062002>
<FONT
face=Arial><SPAN
class=875075116-20062002>Appreciate your
courage
<FONT
face=Arial><SPAN
class=875075116-20062002>
<FONT
face=Arial><SPAN
class=875075116-20062002>William
<FONT
face=Arial><SPAN
class=875075116-20062002>
<FONT
face=Arial><SPAN
class=875075116-20062002>
<FONT
face=Arial color=#0000ff size=2>
<FONT face=Tahoma
size=2>-----Original Message-----From: Avcinci
[mailto:avcinci@xxxx]Sent: Wednesday 19 June, 2002
07:12 PMTo: amibroker@xxxxxxxxxxxxxxxSubject:
Re: [amibroker] Trends, random series
William (and
Richard),
Let me try to explain
Leo’s scrambler. It was developed by Tushar Chande from his book
Beyond Technical Analysis, 2nd ed. Chande randomly rearranges thedata
of a ticker to create new sequences. He does this as follows: he
observes the relationship between the O, H, L, and C of the
2nd bar by using the C of the 1st bar as
reference. So, he writes the relationship as:
DeltaO = O –
ref(C,-1);
DeltaH = H –
ref(C,-1);
DeltaL = L –
ref(L,-1);
DeltaC = C –
ref(C,-1);
He samples with
replacement with these formulas and creates patterns that bear the
market’s signature as defined by relative price relationships. The
next step is to use a random number generator to scramble the bars.
When you have a new sequence, you need a starting point, which is
usually the prior C. The new bar is derived from the prior C as
follows (where the Syn prefix stands for the new synthetic values):
Syn-C = ref(C,-1) +
deltaC;
Syn-H = ref(C,-1) +
deltaH;
Syn-L = ref(C,-1)
+deltaL;
Syn-O=ref(C,-1) +
deltaO;
He calculates the
interbar relationships as defined above for a ticker of your choice.
He uses a random number generator to pick a number from 1 to x (x
determined by the no. of bars you want to scramble). That number is
the next bar of the sequence. Suppose on the 10th pick, you
pick bar 5. Then the original bar 5 becomes bar 10 of the new
sequence. The bars may repeat. You can generate as long a sequence as
desired. You use the synthetic values determined by the equations
above to establish the next bar’s appearance. Thus, what you wind up
doing is encapsulating the market behavior in the original bar 10and
reproducing it in another sequence to create new synthetic data. You
can generate a variety of chart patterns of any length using data
scrambling. According to Chande, you can generate 100 years of data
and test your system against a variety of market conditions. Since
these are the types of patterns you are likely to see in the future,
this is the most rigorous out-of-sample testing you can achieve.
The above commentary
was partly plagiarized from Chande’s book, and it is merely a
synopsis. To get a clearer understanding of the methodology with
spreadsheet examples, you should read his chapter devoted to
scrambling.
I have attached 2 gifs
showing what scrambled data look like on 2 different tickers. Note
that the patterns are not any more unusual than those of normal chart
patterns, and you can see definite trends developing. In fact, ifyou
look at recent real charts of the market indices, you will see many
instances of V-tops and V-bottoms, which is what William was concerned
about earlier today. Hope this helps. Many thanks to Leo Timmermans
for programming Chande’s scrambler for Amibroker. I asked him recently
if he could program it in such a way to create synthetic data foran
entire watch list at one time. That might be tricky, but he said he
would look into it.
Al V.
<BLOCKQUOTE
>
----- Original Message -----
<DIV
>From:
<A title=wpeters_1@xxxx
href="">wpeters_1
To: <A
title=amibroker@xxxxxxxxxxxxxxx
href="">amibroker@xxxxxxxx
Sent: Wednesday, June 19, 2002
1:22 PM
Subject: RE: [amibroker]
Trends, random series, etc : was MetaStock and AmiBroker
<FONT face=Arial color=#0000ff
size=2>AL,
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>With randomised data isn't there a chance that the so-called
'trend' can reverse on the next tick 'more-so' than data based on
human intervention. And that includes mechnical systems (ie. if
enough traders started using a certain
indicator).
<FONT face=Arial color=#0000ff
size=2>The obvious predictability in human behaviour is what can
give you the little advantage and this is not in totally randomised
data.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>Surely if so your 'randomised' data 'trend' does not have an
equal chance of reversing on each and every tick then its not
randomised.
<FONT face=Arial color=#0000ff
size=2>
<FONT face=Arial color=#0000ff
size=2>Very interesting topic.
<FONT face=Arial color=#0000ff
size=2>William
<FONT
face=Tahoma size=2>-----Original Message-----From: Al
Venosa [mailto:avcinci@xxxx]Sent: Wednesday19
June, 2002 10:54 AMTo:
amibroker@xxxxxxxxxxxxxxxSubject: Re: [amibroker]
Trends, random series, etc : was MetaStock and
AmiBroker
Richard,
I'm glad you took the liberty of changing the subject line. I
don't think you are being a pessimistic, grumpy old man (sorry,
don't know how old you are). Let me throw in another 2 cents into
the discussion of trends. You said in order to have a trend, there
must exist some information in the immediate past that would cause
the trend to persist for a long enough time to profit from the
move. I cannot agree more. That is absolutely true. A bad earnings
report comes out on INTC, and the market immediately reacts by
selling. This is further fueled by some nitwit analyst who changes
his buy recommendation to hold or sell (of course, always after
the fact!). Then, all the other individual "investors" climb
aboard and do more selling, driving the prices still further down.
Then, the company foretells that over the next several quarters
there will be more of the same in regards to diminished sales,
inducing further pessimism that drives the prices still further.
All of this information results in a downward trend. This does not
nor cannot happen with random events like coin tosses. So,
technically speaking, you are perfectly correct in your assertions
.
Now, the trend follower comes in. He couldn't care less what is
driving the prices down. In fact, he purposefully never pays
attention to news events and announcements that may drive prices
in one direction or another. He doesn't care about the cause of
the trend. All he does is climb aboard the resulting trend, using
whatever entry he has fashioned to enable him to take advantage of
this 'breakout' and whatever exit that enables him to exit when
the trend falters. Who was taking the other side of the market
when Neeson was loading up on Nikkei futures, driving the Barings
Bank into bankruptcy? The trend followers who recognized whatwas
happening and capitalized. What I'm saying is, the trend follower
buys or sells all breakouts (however you want to define the term
'breakout') and hopes that the resulting price behavior continues
long into the future. With the scrambler, these 'trends' can
indeed take place even with randomized data, as you have already
pointed out. What I'm suggesting is that your system of entries
and exits doesn't have a clue what caused the breakout to occur
because it's nothing more than a mathematical algorithm that
trades based on certain signals programmed by the user when
certain price behavior occurs. I contend you can test your system
on such price behavior just to learn how well the system reacts to
price breakouts and to see if, indeed, it is effective in finding
trends in the data, albeit randomized data. By the way, these
comments are not limited to trend following systems. I chose that
trading approach merely as an example to make my point.
Richard, I'm not trying to be argumentative on this topic.I'm
just pointing out why I think there might be value in testinga
fully optimized trading model on randomized data. Thanks for the
provocative discussion.
Al V.
>From: "Richard Alford"
>Reply-To: amibroker@xxxxxxxxxxxxxxx
>To:
>Subject: [amibroker] Trends, random series, etc :
was MetaStock and AmiBroker
>Date: Wed, 19 Jun 2002 08:35:15 -0500
>
>I agree on the oxymoronic nature of so called
random series. The problem results from using very particular
distributions for the random numbers which allows the
mathematicians to write books and teach classes. The Gaussian
functions are one of the most well-behaved functions around.
>
>wrt trends in scrambler and other "random" time
series - simply stated if they are truly random there is
absolutely no information about the past in the next event. Your
coin toss observation is a great example: given the 1/1000 case of
10 heads in a row the odds of another head are precisely 50% for
the next toss.
>
>In order to discuss a "trend" there must be some
information that persists. One can generally, perhaps always,see
trends in a random series - in hindsight - however it contains no
predictive information. There are systems that seek to identify
trending vs. trading range behavior (I think Ehlers addressesthis
issue - can't say with how much success), however, they are
inaccurate at the changes. A moving average on the coin toss
series will identify trends - but the lag in the moving average
that we all know of and try to remove is the "hindsight" problem.
>
>Trend scale is still another issue. The apparent
fractal behavior of the stock market suggests that one can find
trends, and/or search for trends, over any time scale desired.
(The inflationary or perhaps evolving nature of economies have
placed a long term upward bias on the trends that can confusethe
issue.) Once again a moving average of any length on a random
series will display periods of upward and downward "trends".
>
>Sorry to be such a pessimistic old grump.
>
>Cordially,
>
>Richard
>
> ----- Original Message -----
> From: Avcinci
> To: amibroker@xxxxxxxxxxxxxxx
> Sent: Tuesday, June 18, 2002 8:19 PM
> Subject: Re: [amibroker] Re: MetaStock and
AmiBroker
>
>
> Richard,
>
> >>distribution of the trends is predictable
for a random series >> This sounds oxymoronic to me, i.e.,
predictable distribution of random trends. But, I'll take your
word for it. :-))
>
> Regarding the scale when speaking of trends, I was
referring to weeks to months, perhaps even a year or two at most
(stocks, not futures), but not decades. This brings us back to
discussing the Scrambler. You said you don't hold much if any
value in it, but can you elaborate just a little more on whatyou
mean in terms of the scale I am referring to, because in a period
of 130 trading days, I noticed some pretty decent trends develop
that lasted long enough to be of value in system testing? Thanks,
Richard. By the way, I agree that this discussion, interesting as
it is, has absolutely nothing to do with Metastock!!
>
> AV
>
> ----- Original Message -----
> From: Richard Alford
> To: amibroker@xxxxxxxxxxxxxxx
> Sent: Tuesday, June 18, 2002 3:55 PM
> Subject: Re: [amibroker] Re: MetaStock and
AmiBroker
>
>
> wrt rsi/stochastics/cmo/etc... my point was that
they attempt to identify a phenomenon that has been observed as an
indication of overbought/oversold. There is a reasonable belief
that the arithmetic created to identify that behavior can be a
useful indictor of that condition. The comment offered for Elliot
wave and Fibinacci behavior also points to a large community that
believes that there is a hidden structure to the market and human
behavior - I am not a devotee of that belief, but I also havemy
doubts about Madam Cleo....
>
> wrt trends in random numbers: there is, of course,
a trend between any two non=equal numbers. Given a suitable
sample, the distribution of the trends is predictable for a random
series - probably also random and if the normal Gaussian (pardon
the pun) distribution is used, as the derivative (or instantaneous
slope/trend) is a skewed Gaussian. Interesting to the
statisticians but not very useful to the average investor.
>
> wrt: trending 30% - that really has to be a matter
of scale. If you chose a suitably long moving average you can
probably even consider today as part of a long time upward trend -
not particularly useful - although even my meager portfolio looks
better today than it did in 1970 - not saying much....
>
> Interesting discussion - don't have the faintest
idea what it has to do with MetaStock :)
>
> Cheers,
>
> Richard
>
>
> ----- Original Message -----
> From: Al Venosa
> To: amibroker@xxxxxxxxxxxxxxx
> Sent: Tuesday, June 18, 2002 2:28 PM
> Subject: Re: [amibroker] Re: MetaStock and
AmiBroker
>
>
> Thanks, Richard. I always enjoy your mathematical
insights. I hope you didn't get the impression I was suggesting
'optimizing' on random bars. I was merely suggesting 'testing'
your already optimized system on the scrambled data, seeing if it
can detect those occasional occurrences of non-random behavior of
random numbers, that's all. Certainly if a trend-following system
can detect trends in random price bars, it ought to be able to
detect them in real, non-random price bars, too, don't you think?
Markets trend only about 30% of the time, as I am told, so
wouldn't you consider non-trending markets (i.e., sideways
markets) somewhat representative of random price behavior?
>
> I also get the impression you are not an advocate
of overbought/oversold oscillators, right?
>
> Al Venosa
> avcinci@xxxx
> >From: "Richard Alford"
> >Reply-To: amibroker@xxxxxxxxxxxxxxx
> >To:
> >Subject: Re: [amibroker] Re: MetaStock and
AmiBroker
> >Date: Tue, 18 Jun 2002 12:55:33 -0500
> >
> >The non-random appearance of random numbers is
a very well know phenomenon - probably accounts for the continued
enthusiasm in slot machines in Vegas...
> >
> >I stand by my comments. In particular, the
various oversold/bought indicators rely on the observation that
the close tends to be higher in the l-h range in the overbought
condition - the persistence of such a trend is interpreted as
buy-sell behavior.
> >
> >I personally see no use in random data. There
is no information in random data and no expectation of the past
affecting the future - the only reason for technical analysis
(other than too much time on one's hands) is to glean information
about the near future. There are, of course, degenerate uses for
random numbers in testing, but to test, build, or, heaven forbid -
optimize, using random time series is pointless at best.
> >
> >wrt coin tosses: the coin is a Markov process
of length 0 - no memory. If the market is the same, we may aswell
head for Vegas and get free drinks as we go broke. (OK - the
market MAY be a fair game, but I doubt it.)
> >
> >Just some thoughts...
> >
> >Richard
> > ----- Original Message -----
> > From: Al Venosa
> > To: amibroker@xxxxxxxxxxxxxxx
> > Sent: Tuesday, June 18, 2002 9:57 AM
> > Subject: Re: [amibroker] Re: MetaStock and
AmiBroker
> >
> >
> > Well, Richard, to be honest, I haven't yet
done any back (er, I mean forward) testing of the random timebars
created by Scrambler. However, I have looked at the results of
randomly scrambling the bars and watching them form on the screen,
and you can see definite trends develop that can last several
months, much like in real life. So, at least theoretically, it
seems to me that, if you are testing a short-term or
intermediate-term trend-following system, it ought to be ableto
pick up on those short, seemingly non-random trends (even though
they were developed from a random number generator). Remember, if
you flip an honest coin 1000 times, there is a low but finite
probability that it will come up heads 10 or 12 or 15 times in a
row.
> >
> > AV
> >
> >
> > >From: "Richard Alford"
> >
> > >Reply-To: amibroker@xxxxxxxxxxxxxxx
> > >To:
> > >Subject: Re: [amibroker] Re: MetaStock
and AmiBroker
> > >Date: Tue, 18 Jun 2002 09:48:35 -0500
> > >
> > >I understood that the underlying premise
of technical analysis was that there IS information in the
price-volume behavior. If you test against random data the results
had better result in random behavior - perhaps mean behavior is
more accurate.
> > >
> > >Perhaps I am delusional, however, I would
appreciate insight into the value of random time bars?
> > >
> > >Cordially,
> > >Richard
> > > ----- Original Message -----
> > > From: Al Venosa
> > > To: amibroker@xxxxxxxxxxxxxxx
> > > Sent: Tuesday, June 18, 2002 9:23 AM
> > > Subject: Re: [amibroker] Re: MetaStock
and AmiBroker
> > >
> > >
> > > Sorry, Dimitris, I thought you knew
about it. Go to post 19331 and download it, following Leo
Timmerman's instructions. What he has done is created a VBscript
tool based on Tuchar Chande's price bar scrambler that scrambles
all the bars of a given ticker from the previous x bars (barsand
ticker set by the user) and then re-orders those bars randomly
into the future, thus creating a ticker that you can use in
forward-testing. If you set the no. of bars to, say, 130, youcan
generate 6 months worth of new data for each ticker you do this
on. Very, very cool. It differs from William's Simulator in that
it actually generates new data rather than repeating existingdata
from the last x bars. Try it, you'll like it.
> > >
> > > Al Venosa
> > >
> > > >From: "dtsokakis"
> > >
> > > >Reply-To: amibroker@xxxxxxxx
> > > >To: amibroker@xxxxxxxxxxxxxxx
> > > >Subject: [amibroker] Re: MetaStock
and AmiBroker
> > > >Date: Tue, 18 Jun 2002 13:49:30
-0000
> > > >
> > > >What is Leo's scrambler ??
> > > >DT
> > > >--- In amibroker@xxxx, "Al Venosa"
wrote:
> > > > >
> > > >
> > >
> > >
> >
>------------------------------------------------------------------------------
> > > Chat with friends online, try MSN
Messenger: Click Here
> > >
> > > Your use of Yahoo! Groups is subject to
the Yahoo! Terms of Service.
> > >
> >
> >
>
>------------------------------------------------------------------------------
> > Get your FREE download of MSN Explorer at
http://explorer.msn.com.
> >
> > Your use of Yahoo! Groups is subject to the
Yahoo! Terms of Service.
> >
>
>
>--------------------------------------------------------------------------
> Join the world's largest e-mail service with MSN
Hotmail. Click Here
>
>
> Your use of Yahoo! Groups is subject to the Yahoo!
Terms of Service.
>
>
>
> Your use of Yahoo! Groups is subject to the Yahoo!
Terms of Service.
>
>
> Yahoo! Groups Sponsor
> ADVERTISEMENT
>
>
>
>
> Your use of Yahoo! Groups is subject to the Yahoo!
Terms of Service.
>
Send and receive Hotmail on your mobile device: <A
href="">Click
HereYour use of Yahoo! Groups is subject to the <A
href="">Yahoo! Terms of
Service. Your use of Yahoo!
Groups is subject to the <A
href="">Yahoo! Terms of
Service. Your use of Yahoo! Groups
is subject to the Yahoo!
Terms of Service. Your use of Yahoo!
Groups is subject to the <A
href="">Yahoo! Terms of
Service. Your use of Yahoo! Groups is subject
to the Yahoo! Terms of
Service. Your use of Yahoo! Groups is
subject to the Yahoo! Terms of
Service. Your use of Yahoo! Groups is
subject to the Yahoo! Terms of
Service. Your use of Yahoo! Groups is
subject to the Yahoo! Termsof
Service.
|