|
PureBytes Links
Trading Reference Links
|
This is off topic to actually trading, but there are probably a few
folks on the list that would have some input which would help me address
a design issue for a data repository (www.ufdb.com).
Keeping track and applying stock-splits is error prone, which is why
several data vendors prefer to store data in its raw format and split
information separately. However, adopting this approach for an on-demand
historical data serving implies that before split adjusted data could be
retrieved, splits would have to be run against it. That's compute
intensive if done on the fly when there are several users retrieving
data almost simultaneously, and resources could be overwhelmed.
Deploying a computer (servers) that address the level of capability
needed to make any delay in on-demand retrieval of on the fly
split-adjusted data transparent probably isn't an economic option given
the cost of software licensing-Enterprise versions of Windows Server and
SQL Server, per processor. (clustered 64 bit quad processors, for
example). So a design paradox arises-compromising error proneness and
split-adjustment.
The solution seems to be to store historical data in both formats-split
adjusted as of today and raw data. However, implicitly that doubles the
size of a db (for stocks), which has flow on effects to local,
redundant, colocated and backup storage, bandwidth usage, and
performance if all of the tables are on the same computer.
In short, if you had to make a decision about the above, what would you
do?
Thanks in advance for any suggestions
Colin West
|