|
PureBytes Links
Trading Reference Links
|
Justin,
At 07:16 AM 1/12/2004 -0800, you wrote:
Delivery will be via IP allowing global customer base and I would like
to see some functionality that is not currently widely offered, such
as client acknowledgement of every tick, ability to request historical
data (to recover client downtime) and ability to detect & re-transmit
corrupted (in transit) data to name just a few.
I was with you until this paragraph, when I didn't know whether to double
over laughing or start shedding tears. The line that really got me was
"client acknowledgment of every tick". [re-transmission and data recovery
are reasonable enough requests].
I've worked with a fair number of commercial messaging systems, including
Talarian (powering 65% of all US exchanges), and TIBCO (who decided to
acquire Talarian and has the 2nd largest share of US exchanges), to IBM
MQSeries, a number of JMS providers, and other proprietary techniques.
Talarian smart-sockets is far and above the lead performer in terms of
high-volume, reliable messaging. Application of TIBCO rendezvous is
extremely sensitive to how your network is tuned for segmentation and
network traffic.
Typically most publish/subscribe mechanisms used for market data are
one-way (a kind of "directed broadcast"). The reliability of a one-way
mechanism will be related to the messaging software, networking, and
hardware. There is tremendous variation in quality, and here is where
you're likely to get the biggest bang for your buck.
There are several levels of quality in "guaranteed message delivery". It's
a sure bet that this level of service will double the traffic level on the
client connection, so you will demand more of the pipe. The server,
however, is where costs become prohibitive, since it now must deal with
both reliably sending messages, and acknowledging messages from every
single client. So, for 100 clients you now have roughly 100x the volume on
the server. Do that for every subscriber, every tick, on every symbols and
... well, you get the idea.
Institutional players that require high service levels (not all of which
require acknowledgement) typically employ server farms to scale enough to
handle what I've just described. Often they use pipes that are dedicated
private connections; often in duplicate or triplicate.
I think existing data vendors have plenty of room for improvement. Many
improvements can be handled in a cost-effective manner. Some of the
improvements you suggest will be cost prohibitive.
I do not accept that in 2004, we can place a robot probe on Mars that
is sending back 3D colour images, but cannot disseminate live pricing
from a first world exchange, over a first world telecommunications
infrastructure to an end a customer without the data getting lost or
corrupted while in transit.
I couldn't agree more. Just remember the price tag for that little Mars
excursion; and you may think twice about placing that tab on your Visa.
It's always about trade-offs.
Kevin
|