|
PureBytes Links
Trading Reference Links
|
well, i sure hope that the new up coming super platform
will have bad tick filtering on the level...
then we would not have to worry about the issue anymore.
with data 99.99% real time clean we can all breath easier...
let's cross the fingers... and the toes...and sit still.
until then we can only speculate, ponder or use partial
solutions. the best you can do is to understand bad ticks in detail,
for example, examine in detail how bad ticks affect
TS 6.0 order 1. model variable computation, 2. signal generation,
3. real time execution. if you dig into it you can find the
best possible solution given the current 6.0 platform constraints...
in fact, funny as it may sound, the best solution to 6.0 bad tick
problem is to let the system execute the orders on bad tick and get a refund on comm
instead of ignoring the order and getting your strategy out of synch with your position.
if you know how to handle bad ticks in 6.0 you can set up the system
1.catch bad ticks, even on bar data, 2. exclude bad ticks from model variables calculations
3. generate optimal order execution bad tick response strategy that
minimized the loss... 4. get your money back from TS at least for commissions...
( the last one is especially important, because if we all do that they will have
no other choice but to clean up the data... currently they pass the bad tick
responsibility on the exchange which is baloney, which is illegal because they
provide: data, strategy platform and execution so they should be fully responsible
for errors in any of the three parts... if there is error in execution due to
error in data and not in your system, THEY ARE RESPONSIBLE!!!
if the system generates an order to buy at the price that does not exist or
exists but way outside of the current market, THEY ARE RESPONSIBLE!!!
TO SAY THAT I PROVIDE AUTOEXECUTION PLATFORM BUT NOT
RESPONSIBLE FOR AUTOEXECUTION ERRORS DUE TO DATA ERRORS
IS NONSENSE, shifting responsibility on the exchange is bullshit because the
exchange will tell you that it's your resposibility to clean up data errors before you
trade!!! )
however based on my own research bad tick affect can be minimized even under
TS 6.0 to a certain degree, ie it will cost you but what it will cost you will be minimized.
if you can prove that the order got executed due to bad tick you should get your money
back for the trade... no other way around it...
i have an in detail write up on how to properly handle bad ticks in TS 6.0,
if anyone is interested let me know.
bilo.
ps.i don't think you can hack into TS and
do it because 5.0 keeps as i understand bar data in 1 and 5 min increments...
so if you want to stick a filter between the server and the chart ( which is the
best way ) it might not work because charts poll and compile data from 1 min resolution database...
not from tick database ( where you should do filtering )...
in 6.0 it's impossible because bar data is transmitted as o,h,l,c bars.
the only way a real time tick filter can be implemented is by using tick chart...
but here you got to load a shitload of data into the chart, ie you need a 10Ghz machine to
run 10 100 day ticks charts with tick filters.
there is another alternative - f..ck nazdaq ( low volatility anyway now )
and trade whatever data is clean... take your business to where data is clean...and away from
dirty pits and crappy feeds...
----- Original Message -----
From: "Brian" <blink64@xxxxxxxxx>
To: "List, Omega" <omega-list@xxxxxxxxxx>
Sent: Friday, March 15, 2002 11:29 AM
Subject: RE: Tick data - data scrubbing
> A couple responses have come back defending sterile-clean data.
>
> Based on my experience with at least 4 different data vendors over the years
> and running systems on each one, I've been forced to draw these conclusions.
>
> Most systems (even simple ones) are tpyically affected by even small
> differences in the data. The same system run on say DTNs data will yield
> different signals and results than say TRAD's or Quote.com's. In some cases
> the differences will be marignal and in others, the differences will be
> significant.
>
> Most people don't have a tick scrubbing algorithim built into TradeStation
> unless they've custom hacked into TradeStation and put one either before
> data gets to TradeStation (bad idea) or between the charting module and the
> server (good idea). So to say this is part of step 2 of the process, is
> implying that TradeStation users are not completely using good system
> testing and trading methods because they don't have a tick filter, which all
> users who responded agreed is needed.
>
> Thus, for 99.99999% of the people out there using TradeStation, we either
> have to work around the problem (maintain your own data or use market only
> signals or other creative techniques) or recognize that in some cases (not
> futures as much as stocks), bad ticks are a way of life we have to deal
> with.
>
> Recognizing that different data streams and quality of data produce
> deifferent results (which is probably why so many traders are adamit about
> capturing every last tick published by the exchange and why they avoid feeds
> like BMI and inisisting on the fastest serial ports and so on) it stands to
> reason that merged, cajoled and otherwise sterile-clean data that is unlike
> anything your TradeStation will ever see will yield results that will be
> hard if not impssible to duplicate using TradeStation and your real time
> feed.
>
> There is definitely value in historical data but I think the value is
> reduced when someone else cleans it. I would rather get the raw data, apply
> my own filters, and then hook that same filter up to my trading platform so
> my real time data matches my historical data as much as posssible.
>
> Brian.
>
> -----Original Message-----
> From: Bob Fulks [mailto:bfulks@xxxxxxxxxxxx]
> Sent: Friday, March 15, 2002 6:37 AM
> To: Brian
> Cc: List, Omega
> Subject: RE: Tick data - data scrubbing
>
>
> At 11:24 PM -0800 3/14/02, Brian wrote:
>
> >You know I was out at a site that sold tick data and was reading
> >about how they scrub their data from head to toe. This makes no sense
> >to me...
>
> <snip>
>
> >A far better solution, IMO, is to maintain your own data from your
> >vendor or the raw data, apply some general tick cleaning scrubbers
> >(like ticks that fall 2% outside previous tick and so on) to an ASCII
> >data file, and use that to do your backtests.
>
>
> Consider trading system development as a two-step process:
>
> Stage 1: Find some market characteristic that is tradable
>
> Stage 2: Convert this to a robust, tradable system.
>
> During Stage 1 you need to test lots of things with minimum effort so
> you need clean data and "prototype" trading system code - lots of
> simple code segments, pre-canned functions, etc.
>
> During Stage 2 you need to streamline the code, add bad-tick filters,
> error checking, adaptive parameters, code to handle special
> exceptions, etc., to make the system work with what it will actually
> encounter in real trading with real data.
>
> Any experienced software developer will tell you that the two cases
> are very, very different...
>
> Bob Fulks
>
>
>
>
|