|
PureBytes Links
Trading Reference Links
|
Treliff and Brian
I've got more work to do to understand what's in the one years subscription
of SIPro with multiple CDs but
here's an interim reply on keys...they are the field labels, or the column
headings. Databases
use keys with which to organize the data and the export function outputs
both the
keys in a small file and the data in a much larger file in one step. BTW
you can choose the
database, portfolio or screening subset to export.
For example the keys are the XLS column headings but in a separate file.
Company Name - Ticker - Exchange - SMG - IMG - Cash Flow Growth 12m -
Sales Growth 12m...... Sales Growth 7yr
I thought you had a separate Fundamentals forum but I can see where the
files are being placed.
Planning to load more than one CD sequentially to see if a new month's
version updates common fields or
if it adds additional history to the set.
Best regards
JOE
----- Original Message -----
From: "treliff" <treliff@xxxxxxxxx>
To: <amibroker@xxxxxxxxxxxxxxx>
Sent: Sunday, July 16, 2006 10:33 PM
Subject: [amibroker] Re: Historical Fundamental Data
> NOTE:
>
> I just wanted to submit a reply to Brian and Joe's earlier posts when
> I noted Brian's newest. Allow me to post mine anyway with apparently
> some similar requests to Joe re. SI Pro.
>
> Brian thanks for your gargantuan effort in listing all the sites.
> Forgive me though for pointing out that the heading:
>
> 1. most likely sites for exportable historical fundamental data
>
> seems very optimistic. I have not visited all sites but many I know
> and know OF that they do not include hist. fundas, at least not on
> individual stocks (for example tradestools I just visited has index
> data only). SI Pro still seems the best bet so far (for US stocks).
>
> Anyway my original reply:
>
> Joe, thanks very much for explaining about SI Pro export. I'm not
> sure what is a "key", is it one of the main categories like
>
> Balance Sheet (Annual)
> Growth Rates
> Multiples
>
> as used in their field list
>
> http://aaii.com/stockinvestor/programinfo/SIProFieldList.pdf
>
> If I counted correctly there are 20 of these "categories" and you
> mention 22 tables/files so I assume one can export one category at a
> time. Still not clear exactly what a "key" is. If indeed you could
> post some in the files section that would be great.
>
> One other thing, if I may. On the AAII site it says:
>
> "Quarterly data is available from the last eight quarters and annual
> data spans the last seven years"
>
> Does that mean that every separate (monthly) CD contains data back as
> in the quote above? Or does a monthly CD only contain the past
> month's data?
>
> Joe sorry for requests piling up on you. I guess we could just go for
> the money back guarantee:
>
> "Test Stock Investor Pro on your PC, for as long as a year, before
> you decide if it is right for you."
>
> but without further explanation. I assume one pays the full yearly
> fee and if not satisfied, try to get reimbursed. Seems like a great
> deal, too good to be true really, so probably not as easy as it
> sounds indeed. (I'd be the last to ask money back if the service is
> indeed what I expect, let me get that straight :)
>
> Brian, re. "kicking out the previous value" indeed I mean that at
> most sites only the current (most recent) value is available. With a
> QP subscription there is no way to trace last April's short interest
> on the stock (just an example), 1st) because the QP site doesn't give
> access to these hist. data (if they have stored it at all), 2nd)
> because AB does not store past values in an array (and btw neither
> does it this with the new Yahoo fundas): yesterday's value is lost
> once today's value is imported (please correct me if I'm wrong).
>
> Now even within these limitations the various sites have their
> nuances. Brian, you might as well leave tbsp out (I use `m for option
> data) because their funda fields are very few and there is even no
> way to export them from their analysis program (and it doesn't store
> historicals, of course).
>
> With Zacks, which I have used since Nov '05 to start building
> historicals (I chose Zacks because of their Zacks rank and upcoming
> earnings dates) I do it the hard way: run a screen, copy/paste the
> data from the web page into excel, run a macro pre-crunching the data
> and shuffling everything into 6 columns (OHLCVI) for AB ascii import
> using good'ol artificial tickers 2_MSFT, 3_MSFT etc. (Let me just
> mention I had to google "SQL" so that's probably not for me...)
>
> Meanwhile I have built a half year's data but I was only able to
> focus on a limited amount of fields because, in combination with a
> large amount of stocks, the webscreener just can't handle anymore. Of
> course it wasn't intended for this kind of use, but I have e-mailed
> Zacks and asked if I could BUY the raw data but nope, they don't
> offer this. (On 2nd thought, after a few months copy/pasting, it
> turns out the data quality is terrible, aarrggh...).
>
> So it's free but sloppy and I'd prefer to pay a fair amount for a
> neat .csv if only historicals were included.
>
> Thanks again All for the discussion so far.
>
> -treliff
>
> --- In amibroker@xxxxxxxxxxxxxxx, "brian.z123" <brian.z123@xxx> wrote:
>>
>> Third time lucky?
>>
>> http://finance.groups.yahoo.com/group/amibroker/files/Fundamental%
>> 20Data/
>>
>> --- In amibroker@xxxxxxxxxxxxxxx, "brian.z123" <brian.z123@>
>> wrote:
>> >
>> > Hello Joe,
>> >
>> > Thanks to you and also treliff for jumping in and putting some
>> > weight on the oars; we have moved a little further up the stream.
>> >
>> > I think there will be a lot more questions about the subject when
>> > the SQL plug-in comes out.
>> > It is all new to me, as it will be to most others.
>> > Prior to this I couldn't find a retail program that would allow
> me
>> > to use fdata inputs in formulas.
>> > Of course, I haven't tried every single program e.g I bypassed
>> > Tradestation as it is 'trading platform' based and since 9/11 USA
>> > regulators have tightened up on foreign investors and TS no
> longer
>> > accepts accounts from US non-residents.
>> > Prior to this I did my funda analysis by manually entering data
> to
>> > Xcel, and that was for the Aussie market only.
>> >
>> > So for the list of sites I have posted there are a lot of
>> questions
>> > to be answered and trefill's question is one of the more
> important
>> > ones.
>> >
>> > 'Will an AAII subscription truly enable me to pour all past data
>> > into a .csv file?'
>> >
>> > It is a question equally as relevant to all the sites.
>> >
>> > The reason I posted the list in the first place was because the
>> > inititial reaction of many would be to go straight for the AAII
>> > version or perhaps similar sources.
>> > By questioning a little further perhaps we will find about the
>> pros
>> > and cons of each site in advance, and in some cases, save
>> ourselves
>> > some time and money.
>> > There are also going to be additional problems with f databases
>> > compared to price databases; price only comprises 6 fields
>> > (OHLCVOI), the fields/formats are standard across all providers,
>> > data is precisely time dependent, data is continuous and
> histories
>> > are complete and deep in most cases i.e apples = apples almost
>> every
>> > time.
>> > The opposite applies to fdata in all five areas.
>> >
>> >
>> > Re your comments.
>> >
>> > No it is not more than we need to know.
>> > It is spot on.
>> >
>> > So far, as I understand it, AAII has 100-200 data fields
> available
>> > with varying number of years available for each.
>> > The screening software will export in a suitable format for AB
> but
>> > it will not export the database in its entirety in one step.
>> > AB users would need to export,say, 5-10 segments of the database
>> > separately and then join them together if they wanted one
> database.
>> > Alternatively they could just keep them as,say, 10 separate SQL
>> > bases.
>> >
>> > If this is correct, it leaves us with a big question mark over
> the
>> > use of the keys (what are they and what are they for) and also
> the
>> > timing of the updates (weekly/monthly are always lagging updates).
>> > There is also a minor question of whether the database will
> always
>> > be 7yrs/8qtrs or is it growing each year?
>> >
>> > The files re this topic are at the Fundamental Data folder; menu
>> at
>> > the top left corner of this page>files?fundamental data.
>> >
>> > I think this will take users to it:
>> >
>> > http://groups.yahoo.com/group/fundamental data
>> >
>> > If you can post a .csv of one table downloaded for as many years
>> as
>> > you can get, with the relevant key, that would be great.
>> > Even screenshots pasted into Word would do.
>> > That would be service 'above and beyond the call of duty' and
>> enough
>> > for anyone contemplating fdata use and AAII in particular.
>> >
>> > From there interested users should move to their own trial via
> the
>> > money back guarentee.
>> >
>> > I have posted a text file there that I am using to keep track of
>> all
>> > fdata providers of any kind.
>> >
>> > Regards,
>> >
>> > BrianB2.
>> >
>> >
>> > --- In amibroker@xxxxxxxxxxxxxxx, "Joe Landry" <jelandry@> wrote:
>> > >
>> > > Brian - A few notes back someone mentioned that SI Pro had a
>> > feature to
>> > > export data. I fired up the dated
>> > > version I have and found that the data is organized in 22
>> > tables/files that
>> > > can be exported one at a time. Also there
>> > > are keys that get exported with each file. If any interest I
>> can
>> > store 2 or
>> > > 3 of these keys in the files section
>> > > of the fundamental forum for folks to consider, if you'll clue
>> me
>> > into the
>> > > link to that forum.
>> > >
>> > > The Standard Table export consists of 6.8 MB in XLS format.
>> > > There are 7968 issues back in 2004
>> > > Formats available are XLS, Lotus, XBase(dbf), fixed length,
>> > comma
>> > > separated, or tab separated fields.
>> > >
>> > > I'm still working on it but what I see are 2 dimensional
>> tables,
>> > ticker
>> > > rows by column data, as a sample
>> > > of the Growth Rates keys table below shows. Looks like the
>> > historical
>> > > aspect is relative coarse grained
>> > > going back 8 years on some fields, but that's expected since
> the
>> > statistics
>> > > are reported quarterly.
>> > > A stumper is how would you get a historical institutional
>> > ownership unless
>> > > you had collected the monthly CDs over
>> > > 8 years. I could be off the mark here, and if so would
>> appreciate
>> > someone
>> > > setting me straight.
>> > >
>> > > Probably more than you'd want to know. Looks to be a lot of
>> work
>> > to
>> > > interface to SQL server, even with
>> > > the SQL Server's data transformation tool.
>> > >
>> > > Best regards
>> > > JOE
>> > >
>> > >
>> > > short_name long_name
>> > > COMPANY Company name
>> > > TICKER Ticker
>> > > EXCHG_DESC Exchange
>> > > SMG_DESC Sector
>> > > IMG_DESC Industry
>> > > CFPS_G1T Cash Flow-Growth 12m
>> > > SALES_G1T Sales-Growth 12m
>> > > SALES_G1F Sales-Growth 1yr
>> > > SALES_G3F Sales-Growth 3yr
>> > > SALES_G5F Sales-Growth 5yr
>> > > SALES_G7F Sales-Growth 7yr
>> > > RSALES_G1T % Rank-Sales-Growth 12m
>> > > RSALES_G1F % Rank-Sales-Growth 1yr
>> > > RSALES_G3F % Rank-Sales-Growth 3yr
>> > > RSALES_G5F % Rank-Sales-Growth 5yr
>> > > RSALES_G7F % Rank-Sales-Growth 7yr
>> > > SALES_G1Q5 Sales-Growth from Q5 to Q1
>> > > SALES_G2Q6 Sales-Growth from Q6 to Q2
>> > > SALES_G3Q7 Sales-Growth from Q7 to Q3
>> > > SALES_G4Q8 Sales-Growth from Q8 to Q4
>> > >
>> > >
>> > >
>> > >
>> > > ----- Original Message -----
>> > > From: "brian.z123" <brian.z123@>
>> > > To: <amibroker@xxxxxxxxxxxxxxx>
>> > > Sent: Saturday, July 15, 2006 9:18 PM
>> > > Subject: [amibroker] Re: Historical Fundamental Data
>> > >
>> > >
>> > > > Hello treliff,
>> > > >
>> > > > Agreed that all historical fundamental data is not created
>> equal.
>> > > >
>> > > > I am looking out for peoples favourites or recommended sites.
>> > > >
>> > > > It is a fair question to ask of the sites that have
>> screen>export
>> > > > software if indeed all fields can be exported in one file and
>> if
>> > so
>> > > > what is the depth (years)?
>> > > >
>> > > > Re your comment 'the habit of kicking out the previous value
>> and
>> > > > simply replacing it with the new one seems to be the norm in
>> > > > fundaland'.
>> > > > Do you mean that many of the historical sites I have listed
>> are
>> > in
>> > > > fact only current data providers?
>> > > > I am trying to understand what you pointed out to us there.
>> > > >
>> > > >
>> > > > Minor comments.
>> > > >
>> > > > I have QP and IQ data as two providers who bundle current
>> fdata
>> > with
>> > > > their price data.
>> > > > There are probably others.
>> > > > It used to be a very short list but new players pop up every
>> day.
>> > > >
>> > > > http://www.qp2.com
>> > > > http://www.iqfeed.net/
>> > > >
>> > > > I had Stockwiz in the short list for hist fdata but the site
>> has
>> > > > changed recently and I couldn't find any reference to it that
>> the
>> > > > public can access.
>> > > > They do talk about fundas but no details are given and the
>> link
>> > to
>> > > > the data site is know a log on restricted access site.
>> > > >
>> > > > I added tbsp to the histo list.
>> > > >
>> > > > http://www.tbsp.com/public/default.htm
>> > > >
>> > > > AAII Stock Investor Pro lists the financials that are
>> available
>> > for
>> > > > 7yrs/8qtrs and has a 'money back period'.
>> > > > Admittedly 'money backs' are not as user frienly as free demo
>> > > > versions.
>> > > >
>> > > > http://www.aaii.com/stockinvestor/intro/guarantee.cfm
>> > > >
>> > http://www.aaii.com/stockinvestor/intro/overview/statementcomp.cfm
>> > > >
>> > > > Brian.
>> > > >
>> > > > --- In amibroker@xxxxxxxxxxxxxxx, "treliff" <treliff@> wrote:
>> > > >>
>> > > >> Thanks to everyone sharing their insights on fundamental data
>> > > >> collection. A small contribution:
>> > > >>
>> > > >> Those interested only in:
>> > > >>
>> > > >> Shares out
>> > > >> Float
>> > > >> Short interest
>> > > >> Insider ownership
>> > > >> Institutional ownership
>> > > >>
>> > > >> may consider shortsqueeze.com. At $28 they allow access to
>> their
>> > > >> historical data (starting only mid 2004) in XL format.
>> > > >>
>> > > >> ... and a small request:
>> > > >>
>> > > >> In joining the search for historical funda's I found AAII
> most
>> > > > likely
>> > > >> to indeed have this ability by their Data Exporting feature:
>> > > >>
>> > > >> " All of the fields found in Stock Investor Pro can be
>> exported
>> > in
>> > > > a
>> > > >> variety of formats, including Excel & Lotus spreadsheets,
>> ASCII
>> > > > tab-
>> > > >> and comma-delimited formats, and dBase format. "
>> > > >>
>> > > >> But does this indeed include ALL historicals? Daily, weekly,
>> > > > monthly,
>> > > >> at whichever frequency each field was updated in the past?
>> > > >>
>> > > >> I'm very suspicious and they don't have a trial. Suspicious
>> > > > because
>> > > >> the habit of kicking out the previous value and simply
>> replacing
>> > > > it
>> > > >> with the new one seems to be the norm in fundaland (QP, tbsp,
>> > > > zacks,
>> > > >> reuters, stockwiz etc etc... and I'm afraid many of the sites
>> > > > Brian
>> > > >> has listed).
>> > > >>
>> > > >> So can anyone confirm (Mike perhaps, who has a
> subscription),
>> > for
>> > > >> example re. short interest (or any other fundamental field):
>> > will
>> > > > an
>> > > >> AAII subscription truly enable me to pour all past data into
>> > > > a .csv
>> > > >> file? No PowerTools or SuperScreens, just good'ol plain raw
>> > data :)
>> > > >>
>> > > >> Stock XYZ short interest:
>> > > >>
>> > > >> Jan '05: ...
>> > > >> Feb '05: ...
>> > > >> .
>> > > >> .
>> > > >> Jun '06: ...
>> > > >>
>> > > >> Thanks for your advice!
>> > > >>
>> > > >> -treliff
>> > > >>
>> > > >> --- In amibroker@xxxxxxxxxxxxxxx, "brian.z123" <brian.z123@>
>> > wrote:
>> > > >> >
>> > > >> > Now you see it, now you don't.
>> > > >> >
>> > > >> > Here it is!
>> > > >> >
>> > > >> > http://betterinvesting.stores.yahoo.net/23303.html
>> > > >> > http://nt3.zacks.com/default.htm
>> > > >> > http://online.barrons.com/public/main
>> > > >> > http://sec.freeedgar.com/
>> > > >> > http://today.reuters.com/news/home.aspx
>> > > >> > http://www.aaii.com/stockinvestor/intro/
>> > > >> >
> http://www.angelfire.com/in/vestinukgrowthshares/cdrefs.html
>> > > >> > http://www.aspecthuntley.com.au/af/home?xtm-
>> > > > licensee=aspecthuntley
>> > > >> > http://www.capitaliq.com/
>> > > >> > http://www.cbs.curtin.edu.au/index.cfm?objectId=1B3273A3-
>> 0C32-
>> > > > 2E2E-
>> > > >> > 4B5B799689FF74FC
>> > > >> > http://www.crsp.chicagogsb.edu/
>> > > >> > http://www.csidata.com/
>> > > >> > http://www.exchange-data.com/products/fundamental_data.aspx
>> > > >> > http://www.fisonline.com/
>> > > >> > http://www.fordequity.com/html/pr_data_usdaily.htm
>> > > >> > http://www.globalfinancialdata.com/index.php3
>> > > >> > http://www.hemscottdata.com/
>> > > >> > http://www.investmenteasy.com/home.aspx
>> > > >> > http://www.it is a capital mistake to theorize in advance
>> of
>> > the
>> > > >> > facts.Sherlock Holmes
>> > > >> > http://www.market-eye.co.uk/
>> > > >> > http://www.marketdata.sungard.com/
>> > > >> > http://www.msodata.com/
>> > > >> > http://www.perfectinfo.com/product_detail.asp?productid=4
>> > > >> > http://www.qaisoftware.com/index.html
>> > > >> > http://www.rimes.com/index.xmp
>> > > >> > http://www.securities.com/
>> > > >> > http://www.thomson.com/index.jsp
>> > > >> > http://www.trackdata.com/
>> > > >> > http://www.tradetools.com/web/scripts/default.asp
>> > > >> > http://www.valueline.com/
>> > > >> > http://www.wvb.com/
>> > > >> > http://www.zacks.com/
>> > > >> > http://www2.standardandpoors.com/servlet/Satellite?
>> > > >> > pagename=sp/Page/HomePg&r=1&l=EN&b=10
>> > > >> >
>> > > >> > Some offer full financials, some are summarised (financial
>> or
>> > > > key
>> > > >> > statistics/ratios) and the third possiblity also applies.
>> > > >> >
>> > > >> > Brian.
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > --- In amibroker@xxxxxxxxxxxxxxx, "brian.z123"
> <brian.z123@>
>> > > >> > wrote:
>> > > >> > >
>> > > >> > > Historical fundamental data providers.
>> > > >> > >
>> > > >> > >
>> > > >> > >
>> > > >> > > Here is a list of links to sites that provide historical
>> > fdata
>> > > >> > that
>> > > >> > > may be of interest to AB users when the SQL plug-in is
>> > > > released.
>> > > >> > >
>> > > >> > > Some of them were reviewed by forum members at the 'New
>> > > >> > Fundamental
>> > > >> > > Data' and 'Institutional Ownership' topics.
>> > > >> > >
>> > > >> > > Most are USA market biased with some global providers
> and
>> a
>> > > >> couple
>> > > >> > > for specific markets (UK and Aus - sorry I don't have
> any
>> > for
>> > > >> > other
>> > > >> > > individual markets).
>> > > >> > > They include retail investor providers, investment club
>> > > >> providers,
>> > > >> > > academic providers (free to accredited people???) and
>> > > >> > institutional
>> > > >> > > level data (some institutional providers package parts of
>> > > > their
>> > > >> > > systems for individual users).
>> > > >> > >
>> > > >> > > The links don't necessarily lead straight to the data;
>> some
>> > > >> > snooping
>> > > >> > > around is required in some cases.
>> > > >> > > It might also contain errors or dead leads.
>> > > >> > >
>> > > >> > > It is not a complete list by any means but includes most
>> of
>> > > > the
>> > > >> > main
>> > > >> > > suspects.
>> > > >> > > Any further leads would be welcomed, including non-
> English
>> > > >> > speaking
>> > > >> > > sites/markets (as long as the post is in English).
>> > > >> > > The criteria for the list is that the database must
>> include
>> > > > some
>> > > >> > > industry standard fundamental data and be exportable in a
>> > > > format
>> > > >> > > that can be used by AB or other similar programs (for
>> those
>> > > > who
>> > > >> > own
>> > > >> > > more than one software package).
>> > > >> > > For anyone who has info readily available; providers home
>> > > >> country,
>> > > >> > > site URL, market, cost, instruments covered, database
>> > length,
>> > > > EOD
>> > > >> > or
>> > > >> > > weekly data etc, download/update method and frequency,
>> data
>> > > >> > fields,
>> > > >> > > CD available, download or intermediatry software needed
>> and
>> > > >> > download
>> > > >> > > formats available is the type of stuff that provides
>> useful
>> > > >> > guidance
>> > > >> > > for forum members.
>> > > >> > >
>> > > >> > > I might make a comparison table, and file it at this
>> site,
>> > at
>> > > > a
>> > > >> > > later date.
>> > > >> > >
>> > > >> > > Brian.
>> > > >> > >
>> > > >> >
>> > > >>
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > Please note that this group is for discussion between users
>> only.
>> > > >
>> > > > To get support from AmiBroker please send an e-mail directly
> to
>> > > > SUPPORT {at} amibroker.com
>> > > >
>> > > > For other support material please check also:
>> > > > http://www.amibroker.com/support.html
>> > > >
>> > > >
>> > > > Yahoo! Groups Links
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > >
>> >
>>
>
>
>
>
>
>
>
>
> Please note that this group is for discussion between users only.
>
> To get support from AmiBroker please send an e-mail directly to
> SUPPORT {at} amibroker.com
>
> For other support material please check also:
> http://www.amibroker.com/support.html
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
>
>
|