|
PureBytes Links
Trading Reference Links
|
>On 03/27/98 15:32:29 Earl Adamy wrote:
>Data quality generally seems to be of little concern to data providers and
>software vendors alike. The first trading software I ever wrote some 15
years
>ago was a data filter program to clean/filter EOD data. It does simple
things
>like checking for 30% gaps on relatively low volume to catch unreported
splits
>and bad prices, checking for volume which is extremely high with relatively
>little price movement, and checking for ohlc relationships which are
improbable.
>Both Equis and Omega downloaders seem to see fit to post whatever garbage
comes
>in although the latest versions of Metastock do provide a report of
probable
>errors. In course of notifying Dial Data of some data errors recently, I
asked
>them why they don't use simple filters to catch this stuff before it goes
out. I
>included the logic for the filters I use. They thanked me for notifying
them of
>the errors and ignored my question. I'm not singling out DD as I've found
>the
>same thing with the half-dozen plus major EOD vendors I've used over the
>years.
>Finally, I can't understand why, when I do see bad data on my charts, I've
got
>to load another piece of software to fix the data instead of clicking on
the bad
>bar and entering the fixes.
>Earl
Earl:
I agree, raw data received requires the use of filters before you can use
it. We work with mutual fund and variable annuity daily data. To clean it
up, we run the filters that you mentioned, plus keep historical dividend
anniversity dates and compare price changes to two standard deviation of the
daily average of all of the funds with the same objectives. We also use
multiple data sources and compare the results for differences.
The Annuity Price Center (which I own, and thus definitely have an interest
in) does run the same filters and tests on its variable annuity price data
for thousands of sub-accounts each day before the information is released to
the public. http://www.annuityprices.com/
SC/TS/MS price file formats, as well many others, are supported.
Jerry Wagner
http://www.flexibleplan.com/
|