|
PureBytes Links
Trading Reference Links
|
Steve, I’m sorry, but I have to
agree with Joe on this issue…
While I can’t quote the specifics
that he quotes, I do this work professionally – I’m a contract Data
Warehouse jock. (Or, “Database Developer”, “Database
Architect”, or whatever my customer wants my title to be.) Please
believe me…
Data management is at least 10 times more
difficult than it first appears. On a complexity scale, AB looks
like a 10 and IS a 10. OTOH, a big database looks like a 1 and but is EASILY
a 10!
Although getting good data is one of my
biggest concerns, I would strongly recommend that Tomasz NOT to get into this
business. If he did, it would massively impact AB development and
support.
Regards,
Dan.
From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf Of Joe Landry
Sent: Friday, January 06, 2006
5:21 AM
To: amibroker@xxxxxxxxxxxxxxx
Subject: Re: [amibroker] Database
Cleanup Script - need three testers
Hi Steve - I must respectfully disagree on
having the Amibroker group provide "Amidata." I can't
think of a venture more fraught with
losses. I've seen FastTrack struggle over 15 years with
keeping data current and I'm now
experiencing Quotes Plus micues and they struggle to keep data
current as far a fundamental metrics and
distributions, splits and current prices. Most recent gaps have to do
with staying current with Profunds
data and distributions for both FastTrack and QuotesPlus. As I'm
sure you've experienced it is a manpower intensive job with so much
opportunities for "oh sh_ts" that I can't see where it would be
profitable. What with all the Friday night and Sat morning hand
fixes; server outages, database changes, etc. ; and selfishly speaking it would
dilute the support in both existing and new functionality in AB.
It will be interesting to hear Tomasz's views on this when he visits Houston next month.
--- Original Message -----
Sent: Thursday, January
05, 2006 8:15 PM
Subject: Re: [amibroker]
Database Cleanup Script - need three testers
Don't think I can help much with this one - I have gone back
to TC2K and Wordens are very good about keeping their DB current with regards
to tickers. BTW, Nirvana Systems( OmniTrader ) has just come out with their
new OmniData/OmniScan combo in OT2006, which also uses Media General
sectors/industries like TC2K - maybe TJ should seriously consider
offering "AmiData" 8 - ) I'm
sure he would be just as capable as anyone of buying wholesale
data and repackaging for it for retail. It might be a nice
steady monthly income, could be stored right in AB's native DB
for optimal speed, would include fundamentals for those that like them,
etc.
Also, your message reminded me that a while ago, you
thought you might write a replacement for the DLL that scraped earnings
release dates from the web. Just curious if you have followed through on that
one yet? If you need any help, I would be happy to help test that one if and
when the time comes !
----- Original Message -----
Sent: Thursday, January
05, 2006 7:09 PM
Subject: [amibroker]
Database Cleanup Script - need three testers
One of the issues that we deal with is bad data from vendors. I
use QP3. I like it and think that they provide a good product for
the price. However, some of their data is not as clean as I'd like.
To resolve this, I took Tomasz's "Cleanup.js" Jscript and
expanded it. It does a fair amount (see below) and improves the
quality of my data. Although I use QP3 for data, the script is NOT
specific to a particular data source. Since it operates only on the AB
database, it should work with any data source or vendor.
This script is NOT and will NOT be for sale, but I'm willing to share
it with this group IF three people will step forward and be willing to test
it. (Since it deletes symbols, it has to be clean.) Note that
I do NOT want you to test it on a live AB database, only a test AB database.
- Deletes Groups of symbols for GroupIDs that you specify (e.g.,
Preferred Stocks", Mutual Funds, etc.)
- Deletes symbols with no quotes or stale (never changing) quotes.
- Deletes specific symbols (from an "AlwaysDelete" text file
list) that you know are bad or do not want. This is an override
list.
- Never deletes specific symbols (from a "AlwaysKeep" text
file list) that you always want to keep - EVEN IF they have no quotes or
are in a "Delete" group. I.e., this is an override list.
- Keeps logs of the overall runs, deleted symbols, deleted no-quotes
symbols, etc.
In my case, I don't want a lot of unused or bad symbols clutering up my
database. I am not interested in the thousands of Mutual
Funds, Preferred Stocks, Unit, and some other non-stock symbols.
But, I am interested in a handful of Mutual fund symbols (about 30-40),
stocks, ETFs and some Indexes.
My script deletes all symbols which meet certain criteria, including
certain groups. So all Mutual Fund (a Group) symbols are deleted
UNLESS they are in the "Always Keep" list. OTOH, QP contains
some old, unused symbols like the "LE**" symbols which are NOT in a
"Delete" Group but I still want to delete them. These are listed
in the "Always Delete" list.
Before testing it this morning, I downloaded 29,000 symbols from
QP into an AB test database. This included Stocks, Indexes,
Sector/Industry assignments and Mutual Funds. After running the
script, 21,000 were deleted.
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For other support material please check also:
http://www.amibroker.com/support.html
SPONSORED LINKS
YAHOO! GROUPS LINKS
|
|