|
PureBytes Links
Trading Reference Links
|
At 10:13 AM -0500 11/2/98, Val Clancy wrote:
>I am writing a little EL routine to filter out bad ticks as well as gaps in
>data, etc.
>
>Bad ticks - spike high, spike low, zero low.
>
<Snip>
>Questions:
>Has anyone wrote anything like this?
>Any suggestions? Any insights?
>
Randy Tareilo wrote a good summary of what he does on this list about a
year ago. I have attached his post below.
Bob Fulks
------
Resent-Date: Sat, 6 Sep 1997 11:13:42 -0700 (PDT)
Date: Sat, 06 Sep 1997 13:02:54 -0700
From: Randy Tareilo <taracent@xxxxxxxxx>
Reply-To: taracent@xxxxxxxxx
Organization: Tarac Enterprises
MIME-Version: 1.0
To: omega-list@xxxxxxxxxx
CC: vibri@xxxxxxxxx
Subject: Re: Bad ticks
Resent-From: omega-list@xxxxxxxxxx
X-Mailing-List: <omega-list@xxxxxxxxxx> archive/latest/9400
X-Loop: omega-list@xxxxxxxxxx
Precedence: list
Resent-Sender: omega-list-request@xxxxxxxxxx
Patrick Mikula wrote:
>
> These questions are for real time traders using mechanical trading
> systems. How do you deal with bad ticks in real time? Wont bad ticks
> corrupt a real time mechanical trading trading system?
Hi Patrick,
Good Questions! Yes, bad ticks can destroy any mechanical trading
system, either intraday, daily, weekly, etc depending upon your data.
If the data you are collecting is not "corrected" by the data provider
or the data feed, then either a) you need to correct yourself, or b)
make your systems work correctly using bad data.
While you might do a lot of a), or your data vendor might do that, you
really must build your systems to work on bad data. If you are using
intraday data, this is a must. The worst possible system you might have
is one that fire's off a buy/sell recommendation, you take the correct
action, then you look at your chart or data feed and the whole issue is
a bad tick, not a true buy/sell recommendation.
There are several methods to filter bad ticks out of your data. The
best method is to do a backward statsical analysis of your data to
determine the maximum change between ticks of data. If the maximum true
change is for a stock like DELL of 3/4 point, then any tick that is
outside that range can be determined as a bad tick and not used by your
system. TS has a built in function in the Server system to highlight
data that is outside a "normal range" of data. This notice is brought
to your attention in one of the Servers data windows but you still must
edit the data field itself to correct or delete that data item.
If you are not using tick data, but lets say one minute or five minute
data, a similar trick can be used to preprocess the the data before you
use it in a system. Again, I like the idea that if the data point in
question is outside a range of expected values for that field, you
should exclude it from consideration.
Lets say a case in point, you are using five minute bars and the stock,
DELL, has a normal maximum move at five minute bars of not more than one
and 1/2 points. If you have built a system using only the highs of the
bar, you should preprocess the data with a range check before deciding
to use that data field. Look at the code below to see such an example
of self correcting data systems.
{code to determine new highs for a five minute bar look back over 20
bars)
inputs: medianrange(1.5);
vars: vhigh(0), vhigh20(0);
if high < high[1] + medianrange then (check for out of range high)
vhigh = high (high is ok, use as a normal high)
else begin
(we must decide what to use as a substitue for the bad tick,
should we use the open, close, or low plus medianrange for the true
high. anyone of these could also be incorrect as it could also be a
one tick five minute bar. Solution one could be to use the last
correct high. Solution two could be if the open and close are not
equal to the false high then use the higher of both of them.
A third solution is to use a combination
of both solution one and two which I shall do.
if close <> high and (close not the same as the high)
close > open then (close is the highest of the two points)
vhigh = close
else
if open <> high and (open not the same as the high)
open >= close then (open is equal or higher than the close)
vhigh = open
else
vhigh = vhigh[1]; (we do have a problem, bad
data, use last correct high)
vhigh20 = highest(vhigh,20); (find the highest high for
the last 20 bars)
etc, etc, etc for finding the corrected low of the bar
I believe that in writing and using intraday system, you must program
around the difficults of bad data and systems that do not self adjust.
If you don't, then your system will be UNTRADEABLE because if is taking
all of these bad commands to buy/sell. A bad system is worst than no
system because you might want to trade a bad system and fail.
Good Luck and Good System Trading
Randy Tareilo
|