Adding to Dan's comments….I was
talking with CSI Data tech support. I asked how often tickers change since
there have been 30-35 changes each day this year. He said this is normal and CSI
has a full-time employee just to keep track of ticker changes. This does not
count splits, dividends or any other db maintenance issues.
By the way, ticker change updates in CSI
Data are currently handled -- during an update -- by adding a new ticker.
This means AB now has two tickers for the same stock. The first appears to have
stopped trading while the 2nd has only one day of data. Splits, on
the other hand, are completely replaced with corrected data. Support told me
they already have code to "fix" this issue and it just needs more
testing. PS: When you do a clean export the data is correct with a single
ticker representing all the data (with the new symbol).
Note: The above is for a database exported to AB
using the combined symbol + csi_num. I don't know what happens if you use csi_num only. It
may work fine since no "change" is really necessary (as the symbol is not included in
this case).
-----Original Message-----
From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf Of Dan Clark
Sent: Friday, January 06, 2006
08:54
To: amibroker@xxxxxxxxxxxxxxx
Subject: RE: [amibroker] Database
Cleanup Script - need three testers
Steve, I’m sorry,
but I have to agree with Joe on this issue…
While I can’t quote
the specifics that he quotes, I do this work professionally – I’m a
contract Data Warehouse jock. (Or, “Database
Developer”, “Database Architect”, or whatever my customer
wants my title to be.) Please believe me…
Data management is at
least 10 times more difficult than it first appears. On a
complexity scale, AB looks like a 10 and IS a 10. OTOH, a big database
looks like a 1 and but is EASILY a 10!
Although getting good
data is one of my biggest concerns, I would strongly recommend that Tomasz NOT
to get into this business. If he did, it would massively impact AB
development and support.
Regards,
Dan.
From:
amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf Of Joe Landry
Sent: Friday, January 06, 2006
5:21 AM
To: amibroker@xxxxxxxxxxxxxxx
Subject: Re: [amibroker] Database
Cleanup Script - need three testers
Hi Steve - I
must respectfully disagree on having the Amibroker group provide
"Amidata." I can't
think of a
venture more fraught with losses. I've seen FastTrack struggle over 15
years with
keeping data
current and I'm now experiencing Quotes Plus micues and they struggle to
keep data
current as far
a fundamental metrics and distributions, splits and current prices. Most recent
gaps have to do
with staying
current with Profunds data and distributions for both FastTrack and
QuotesPlus. As I'm sure you've experienced it is a manpower intensive job
with so much opportunities for "oh sh_ts" that I can't see
where it would be profitable. What with all the Friday night and Sat
morning hand fixes; server outages, database changes, etc. ; and selfishly
speaking it would dilute the support in both existing and new functionality
in AB.
It will be interesting to hear Tomasz's views on this when he visits Houston
next month.
--- Original Message -----
Sent: Thursday,
January 05, 2006 8:15 PM
Subject: Re:
[amibroker] Database Cleanup Script - need three testers
Don't think I can help much with
this one - I have gone back to TC2K and Wordens are very good about keeping
their DB current with regards to tickers. BTW, Nirvana Systems(
OmniTrader ) has just come out with their new OmniData/OmniScan combo in OT2006,
which also uses Media General sectors/industries like TC2K - maybe TJ
should seriously consider offering "AmiData" 8 -
) I'm sure he would be just as capable as anyone
of buying wholesale data and repackaging for it for retail. It
might be a nice steady monthly income, could be stored right
in AB's native DB for optimal speed, would include fundamentals for
those that like them, etc.
Also, your message reminded me
that a while ago, you thought you might write a replacement for the
DLL that scraped earnings release dates from the web. Just curious if you have
followed through on that one yet? If you need any help, I would be happy to
help test that one if and when the time comes !
----- Original Message -----
Sent: Thursday,
January 05, 2006 7:09 PM
Subject: [amibroker]
Database Cleanup Script - need three testers
One of the issues that we deal with is bad data from
vendors. I use QP3. I like it and think that they provide a
good product for the price. However, some of their data is not as clean as
I'd like.
To resolve this, I took Tomasz's
"Cleanup.js" Jscript and expanded it. It does a fair
amount (see below) and improves the quality of my data. Although I
use QP3 for data, the script is NOT specific to a particular data source.
Since it operates only on the AB database, it should work with any data source
or vendor.
This script is NOT and will NOT be for sale, but I'm
willing to share it with this group IF three people will step forward and be
willing to test it. (Since it deletes symbols, it has to be
clean.) Note that I do NOT want you to test it on a live AB database,
only a test AB database.
- Deletes Groups of symbols for GroupIDs that you
specify (e.g., Preferred Stocks", Mutual Funds, etc.)
- Deletes symbols with no quotes or stale (never
changing) quotes.
- Deletes specific symbols (from an
"AlwaysDelete" text file list) that you know are bad or do not
want. This is an override list.
- Never deletes specific symbols (from a
"AlwaysKeep" text file list) that you always want to keep -
EVEN IF they have no quotes or are in a "Delete" group. I.e.,
this is an override list.
- Keeps logs of the overall runs, deleted symbols,
deleted no-quotes symbols, etc.
In my case, I don't want a lot of unused or bad
symbols clutering up my database. I am not interested in the
thousands of Mutual Funds, Preferred Stocks, Unit, and some other
non-stock symbols. But, I am interested in a handful of Mutual fund
symbols (about 30-40), stocks, ETFs and some Indexes.
My script deletes all symbols which meet certain
criteria, including certain groups. So all Mutual Fund (a Group) symbols
are deleted UNLESS they are in the "Always Keep" list.
OTOH, QP contains some old, unused symbols like the "LE**" symbols
which are NOT in a "Delete" Group but I still want to delete
them. These are listed in the "Always Delete" list.
Before testing it this morning, I downloaded
29,000 symbols from QP into an AB test database. This included Stocks,
Indexes, Sector/Industry assignments and Mutual Funds. After
running the script, 21,000 were deleted.
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For other support material please check also:
http://www.amibroker.com/support.html
SPONSORED LINKS
YAHOO! GROUPS LINKS