[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: real-time data



PureBytes Links

Trading Reference Links

"Gerrit Jacobsen" <jrt@xxxxxxxxxx> writes:
>David,
>This problem is propably again related to the way how the server
>stores the data. It doesn't store the data "as is" it stores only
>differences from tick-to-tick. In between they store the real quote
>occasionally. If any difference between two ticks is too big then the
>algorithm stores a wrong price.

The "reference price" is the first tick of the day, stored as a
thirty-two-bit unsigned number - LOTS of range there.  All subsequent
ticks are stored as signed sixteen-bit offsets from that reference.
So, the most serious problem is an error in that reference price,
because it affects every tick of the day.  For word-length
limitations to affect things, the difference from that initial tick
would have to be plus or minus 32768 or greater. 

>All internet hype stocks and other stocks that move in big jumps are
>effected by this problem. The only solution to this problem is to
>make a new category of symbols in the symbols universe (e.g.
>hype-stocks) where the minimum tick is higher than in reality. You
>loose resolution with this method but it eliminates the problem.
>(The recommended setting for Yahoo, ebay, AOL etc. is 100 dollars
>minimum tick :-)

That would have no effect, unless you convinced the data vendors to
pre-process their data to conform to this format.  As it is, the
server just takes the numbers as they hit it, and stores them as
above. The scaling factors in the Symbol Universe then determine how
that data is represented, but not how it's stored. 

A sensible solution would be to have a "biggest reasonable change"
specified in the Symbol Universe data sheets, and to have the server
check each tick as it arrives.  If the tick is out of bounds, either
toss it, or put it in some file to let the user later decide if it
should be included somehow. 

Lots of bad ticks are really good ticks, but are badly delayed, so
they appear at the wrong time on a chart.  There's lots of
opportunity for error between the actual trading and our servers; on
high-volume stocks, CPU overload, either at the vendor, or the
exchange, or within Omega's server, could easily mangle incoming
data. If you're using Win95 or NT, which are sort-of multi-tasking,
you're faced with interesting questions of just how the Omega server
behaves in the face of data overload.  If you're using Win3.1, you
probably just lose ticks. 

Jim


>> David Powell wrote:
>> >I get NASDAQ stocks real-time and would have to spend at
>> >least an hour per day, per stock deleting bad ticks.