[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: Historical Fundamental Data



PureBytes Links

Trading Reference Links

Hello treliff,

Yes, the list is more possibles than probables and I will edit it 
later based on feedback.
Your critique is welcome.

My guess is that the AAII 7yr data = 5 past yearly (2000-2004) + 8 
recent qtrs (2005-2006).

I have only ever done one money back software trial.
I prefer free trials but I badly wanted to test this particular 
software, in fact I fully intended to purchase it.
They had actually desisted from an earlier policy of allowing free 
trials, so I negotiated a money back trial on a personal basis.
At the end of the trial period (30 days?)I changed my original 
decision (one of the main features I needed was missing; a point not 
obvious from the extensive pre-purchase info).
I gave the vendor a specific example of what I wanted to do that the 
program was not capable of doing.
I was apprehensive about that part of the deal all the way through.
The deal was done on a purchase of TradeSim through David Jenyns of 
http://www.meta-formula.com/
David politely and promptly rebated my purchase fee with no 
hesitation.
That is no recommendation of any product but it is a testimony to 
Davids integrity in refunding.
Still, one swallow doesn't make a summer.

I agree with your points re QP,AB,ABYahoo; they are correct.

AB is releasing a new SQL plug-in, with help manual,specifically for 
fdata storage.
I have no prior SQL experience and I am up for it.
It should be easy and AB will really fly for fundamentalists.
A couple of uses are to build a history from the ABYahoo fdata 
downloads by saving to CSV every day or to import other databases, 
like exported Zacks, into SQL and access directly into AB from there.
You won't need to use artificial tickers anymore.

Thanks for your post and sharing your experience.
I now have some more info on Zacks for our listings.

Brian.


--- In amibroker@xxxxxxxxxxxxxxx, "treliff" <treliff@xxx> wrote:
>
> NOTE:
> 
> I just wanted to submit a reply to Brian and Joe's earlier posts 
when 
> I noted Brian's newest. Allow me to post mine anyway with 
apparently 
> some similar requests to Joe re. SI Pro.
> 
> Brian thanks for your gargantuan effort in listing all the sites. 
> Forgive me though for pointing out that the heading:
> 
> 1. most likely sites for exportable historical fundamental data 
> 
> seems very optimistic. I have not visited all sites but many I 
know 
> and know OF that they do not include hist. fundas, at least not on 
> individual stocks (for example tradestools I just visited has 
index 
> data only). SI Pro still seems the best bet so far (for US stocks).
> 
> Anyway my original reply:
> 
> Joe, thanks very much for explaining about SI Pro export. I'm not 
> sure what is a "key", is it one of the main categories like
> 
> Balance Sheet (Annual)
> Growth Rates
> Multiples
> 
> as used in their field list
> 
> http://aaii.com/stockinvestor/programinfo/SIProFieldList.pdf
> 
> If I counted correctly there are 20 of these "categories" and you 
> mention 22 tables/files so I assume one can export one category at 
a 
> time. Still not clear exactly what a "key" is. If indeed you could 
> post some in the files section that would be great.
> 
> One other thing, if I may. On the AAII site it says:
> 
> "Quarterly data is available from the last eight quarters and 
annual 
> data spans the last seven years"
> 
> Does that mean that every separate (monthly) CD contains data back 
as 
> in the quote above? Or does a monthly CD only contain the past 
> month's data?
> 
> Joe sorry for requests piling up on you. I guess we could just go 
for 
> the money back guarantee:
> 
> "Test Stock Investor Pro on your PC, for as long as a year, before 
> you decide if it is right for you."
> 
> but without further explanation. I assume one pays the full yearly 
> fee and if not satisfied, try to get reimbursed. Seems like a 
great 
> deal, too good to be true really, so probably not as easy as it 
> sounds indeed. (I'd be the last to ask money back if the service 
is 
> indeed what I expect, let me get that straight :)
> 
> Brian, re. "kicking out the previous value" indeed I mean that at 
> most sites only the current (most recent) value is available. With 
a 
> QP subscription there is no way to trace last April's short 
interest 
> on the stock (just an example), 1st) because the QP site doesn't 
give 
> access to these hist. data (if they have stored it at all), 2nd) 
> because AB does not store past values in an array (and btw neither 
> does it this with the new Yahoo fundas): yesterday's value is lost 
> once today's value is imported (please correct me if I'm wrong).
> 
> Now even within these limitations the various sites have their 
> nuances. Brian, you might as well leave tbsp out (I use `m for 
option 
> data) because their funda fields are very few and there is even no 
> way to export them from their analysis program (and it doesn't 
store 
> historicals, of course). 
> 
> With Zacks, which I have used since Nov '05 to start building 
> historicals (I chose Zacks because of their Zacks rank and 
upcoming 
> earnings dates) I do it the hard way: run a screen, copy/paste the 
> data from the web page into excel, run a macro pre-crunching the 
data 
> and shuffling everything into 6 columns (OHLCVI) for AB ascii 
import 
> using good'ol artificial tickers 2_MSFT, 3_MSFT etc. (Let me just 
> mention I had to google "SQL" so that's probably not for me...) 
> 
> Meanwhile I have built a half year's data but I was only able to 
> focus on a limited amount of fields because, in combination with a 
> large amount of stocks, the webscreener just can't handle anymore. 
Of 
> course it wasn't intended for this kind of use, but I have e-
mailed 
> Zacks and asked if I could BUY the raw data but nope, they don't 
> offer this. (On 2nd thought, after a few months copy/pasting, it 
> turns out the data quality is terrible, aarrggh...).
> 
> So it's free but sloppy and I'd prefer to pay a fair amount for a 
> neat .csv if only historicals were included.
> 
> Thanks again All for the discussion so far.
> 
> -treliff
> 
> --- In amibroker@xxxxxxxxxxxxxxx, "brian.z123" <brian.z123@> wrote:
> >
> > Third time lucky?
> > 
> > 
http://finance.groups.yahoo.com/group/amibroker/files/Fundamental%
> > 20Data/
> > 
> > --- In amibroker@xxxxxxxxxxxxxxx, "brian.z123" <brian.z123@> 
> > wrote:
> > >
> > > Hello Joe,
> > > 
> > > Thanks to you and also treliff for jumping in and putting some 
> > > weight on the oars; we have moved a little further up the 
stream.
> > > 
> > > I think there will be a lot more questions about the subject 
when 
> > > the SQL plug-in comes out.
> > > It is all new to me, as it will be to most others.
> > > Prior to this I couldn't find a retail program that would 
allow 
> me 
> > > to use fdata inputs in formulas.
> > > Of course, I haven't tried every single program e.g I bypassed 
> > > Tradestation as it is 'trading platform' based and since 9/11 
USA 
> > > regulators have tightened up on foreign investors and TS no 
> longer 
> > > accepts accounts from US non-residents.
> > > Prior to this I did my funda analysis by manually entering 
data 
> to 
> > > Xcel, and that was for the Aussie market only.
> > > 
> > > So for the list of sites I have posted there are a lot of 
> > questions 
> > > to be answered and trefill's question is one of the more 
> important 
> > > ones.
> > > 
> > > 'Will an AAII subscription truly enable me to pour all past 
data 
> > > into a .csv file?'
> > > 
> > > It is a question equally as relevant to all the sites.
> > > 
> > > The reason I posted the list in the first place was because 
the 
> > > inititial reaction of many would be to go straight for the 
AAII 
> > > version or perhaps similar sources.
> > > By questioning a little further perhaps we will find about the 
> > pros 
> > > and cons of each site in advance, and in some cases, save 
> > ourselves 
> > > some time and money.
> > > There are also going to be additional problems with f 
databases 
> > > compared to price databases; price only comprises 6 fields 
> > > (OHLCVOI), the fields/formats are standard across all 
providers, 
> > > data is precisely time dependent, data is continuous and 
> histories 
> > > are complete and deep in most cases i.e apples = apples almost 
> > every 
> > > time.
> > > The opposite applies to fdata in all five areas.
> > > 
> > > 
> > > Re your comments.
> > > 
> > > No it is not more than we need to know.
> > > It is spot on.
> > > 
> > > So far, as I understand it, AAII has 100-200 data fields 
> available 
> > > with varying number of years available for each.
> > > The screening software will export in a suitable format for AB 
> but 
> > > it will not export the database in its entirety in one step.
> > > AB users would need to export,say, 5-10 segments of the 
database 
> > > separately and then join them together if they wanted one 
> database.
> > > Alternatively they could just keep them as,say, 10 separate 
SQL 
> > > bases.
> > > 
> > > If this is correct, it leaves us with a big question mark over 
> the 
> > > use of the keys (what are they and what are they for) and also 
> the 
> > > timing of the updates (weekly/monthly are always lagging 
updates).
> > > There is also a minor question of whether the database will 
> always 
> > > be 7yrs/8qtrs or is it growing each year?
> > > 
> > > The files re this topic are at the Fundamental Data folder; 
menu 
> > at 
> > > the top left corner of this page>files?fundamental data.
> > > 
> > > I think this will take users to it:
> > > 
> > > http://groups.yahoo.com/group/fundamental data
> > > 
> > > If you can post a .csv of one table downloaded for as many 
years 
> > as 
> > > you can get, with the relevant key, that would be great.
> > > Even screenshots pasted into Word would do.
> > > That would be service 'above and beyond the call of duty' and 
> > enough 
> > > for anyone contemplating fdata use and AAII in particular.
> > > 
> > > From there interested users should move to their own trial via 
> the 
> > > money back guarentee.
> > > 
> > > I have posted a text file there that I am using to keep track 
of 
> > all 
> > > fdata providers of any kind.
> > > 
> > > Regards,
> > > 
> > > BrianB2.
> > > 
> > > 
> > > --- In amibroker@xxxxxxxxxxxxxxx, "Joe Landry" <jelandry@> 
wrote:
> > > >
> > > > Brian - A few notes back someone mentioned that SI Pro had a 
> > > feature to 
> > > > export data.  I fired up the dated
> > > > version I have and found that the data is organized in 22 
> > > tables/files that 
> > > > can be exported one at a time.  Also there
> > > > are keys that get exported with each file.  If any interest 
I 
> > can 
> > > store 2 or 
> > > > 3 of these keys in the files section
> > > > of the fundamental forum for folks to consider,  if you'll 
clue 
> > me 
> > > into the 
> > > > link to that forum.
> > > > 
> > > > The Standard Table export consists of  6.8 MB  in XLS format.
> > > > There are 7968 issues back in 2004
> > > >  Formats available are XLS, Lotus, XBase(dbf), fixed  
length, 
> > > comma 
> > > > separated, or tab separated fields.
> > > > 
> > > > I'm still working on it but what I see are 2 dimensional  
> > tables,  
> > > ticker 
> > > > rows by column data, as a sample
> > > > of  the Growth Rates keys table below shows.  Looks like the 
> > > historical 
> > > > aspect is relative coarse grained
> > > > going back 8 years on some fields, but that's expected since 
> the 
> > > statistics 
> > > > are reported quarterly.
> > > > A stumper is how would you get a historical institutional 
> > > ownership unless 
> > > > you had collected the monthly CDs over
> > > > 8 years.  I could be off the mark here, and if so would 
> > appreciate 
> > > someone 
> > > > setting me straight.
> > > > 
> > > > Probably more than you'd want to know.  Looks to be a lot of 
> > work 
> > > to 
> > > > interface to SQL server, even with
> > > > the SQL Server's data transformation tool.
> > > > 
> > > > Best regards
> > > > JOE
> > > > 
> > > > 
> > > >       short_name long_name
> > > >       COMPANY Company name
> > > >       TICKER Ticker
> > > >       EXCHG_DESC Exchange
> > > >       SMG_DESC Sector
> > > >       IMG_DESC Industry
> > > >       CFPS_G1T Cash Flow-Growth 12m
> > > >       SALES_G1T Sales-Growth 12m
> > > >       SALES_G1F Sales-Growth 1yr
> > > >       SALES_G3F Sales-Growth 3yr
> > > >       SALES_G5F Sales-Growth 5yr
> > > >       SALES_G7F Sales-Growth 7yr
> > > >       RSALES_G1T % Rank-Sales-Growth 12m
> > > >       RSALES_G1F % Rank-Sales-Growth 1yr
> > > >       RSALES_G3F % Rank-Sales-Growth 3yr
> > > >       RSALES_G5F % Rank-Sales-Growth 5yr
> > > >       RSALES_G7F % Rank-Sales-Growth 7yr
> > > >       SALES_G1Q5 Sales-Growth from Q5 to Q1
> > > >       SALES_G2Q6 Sales-Growth from Q6 to Q2
> > > >       SALES_G3Q7 Sales-Growth from Q7 to Q3
> > > >       SALES_G4Q8 Sales-Growth from Q8 to Q4
> > > > 
> > > > 
> > > > 
> > > > 
> > > > ----- Original Message ----- 
> > > > From: "brian.z123" <brian.z123@>
> > > > To: <amibroker@xxxxxxxxxxxxxxx>
> > > > Sent: Saturday, July 15, 2006 9:18 PM
> > > > Subject: [amibroker] Re: Historical Fundamental Data
> > > > 
> > > > 
> > > > > Hello treliff,
> > > > >
> > > > > Agreed that all historical fundamental data is not created 
> > equal.
> > > > >
> > > > > I am looking out for peoples favourites or recommended 
sites.
> > > > >
> > > > > It is a fair question to ask of the sites that have 
> > screen>export
> > > > > software if indeed all fields can be exported in one file 
and 
> > if 
> > > so
> > > > > what is the depth (years)?
> > > > >
> > > > > Re your comment 'the habit of kicking out the previous 
value 
> > and
> > > > > simply replacing it with the new one seems to be the norm 
in
> > > > > fundaland'.
> > > > > Do you mean that many of the historical sites I have 
listed 
> > are 
> > > in
> > > > > fact only current data providers?
> > > > > I am trying to understand what you pointed out to us there.
> > > > >
> > > > >
> > > > > Minor comments.
> > > > >
> > > > > I have QP and IQ data as two providers who bundle current 
> > fdata 
> > > with
> > > > > their price data.
> > > > > There are probably others.
> > > > > It used to be a very short list but new players pop up 
every 
> > day.
> > > > >
> > > > > http://www.qp2.com
> > > > > http://www.iqfeed.net/
> > > > >
> > > > > I had Stockwiz in the short list for hist fdata but the 
site 
> > has
> > > > > changed recently and I couldn't find any reference to it 
that 
> > the
> > > > > public can access.
> > > > > They do talk about fundas but no details are given and the 
> > link 
> > > to
> > > > > the data site is know a log on restricted access site.
> > > > >
> > > > > I added tbsp to the histo list.
> > > > >
> > > > > http://www.tbsp.com/public/default.htm
> > > > >
> > > > > AAII Stock Investor Pro lists the financials that are 
> > available 
> > > for
> > > > > 7yrs/8qtrs and has a 'money back period'.
> > > > > Admittedly 'money backs' are not as user frienly as free 
demo
> > > > > versions.
> > > > >
> > > > > http://www.aaii.com/stockinvestor/intro/guarantee.cfm
> > > > > 
> > > 
http://www.aaii.com/stockinvestor/intro/overview/statementcomp.cfm
> > > > >
> > > > > Brian.
> > > > >
> > > > > --- In amibroker@xxxxxxxxxxxxxxx, "treliff" <treliff@> 
wrote:
> > > > >>
> > > > >> Thanks to everyone sharing their insights on fundamental 
data
> > > > >> collection. A small contribution:
> > > > >>
> > > > >> Those interested only in:
> > > > >>
> > > > >> Shares out
> > > > >> Float
> > > > >> Short interest
> > > > >> Insider ownership
> > > > >> Institutional ownership
> > > > >>
> > > > >> may consider shortsqueeze.com. At $28 they allow access 
to 
> > their
> > > > >> historical data (starting only mid 2004) in XL format.
> > > > >>
> > > > >> ... and a small request:
> > > > >>
> > > > >> In joining the search for historical funda's I found AAII 
> most
> > > > > likely
> > > > >> to indeed have this ability by their Data Exporting 
feature:
> > > > >>
> > > > >> " All of the fields found in Stock Investor Pro can be 
> > exported 
> > > in
> > > > > a
> > > > >> variety of formats, including Excel & Lotus spreadsheets, 
> > ASCII
> > > > > tab-
> > > > >> and comma-delimited formats, and dBase format. "
> > > > >>
> > > > >> But does this indeed include ALL historicals? Daily, 
weekly,
> > > > > monthly,
> > > > >> at whichever frequency each field was updated in the past?
> > > > >>
> > > > >> I'm very suspicious and they don't have a trial. 
Suspicious
> > > > > because
> > > > >> the habit of kicking out the previous value and simply 
> > replacing
> > > > > it
> > > > >> with the new one seems to be the norm in fundaland (QP, 
tbsp,
> > > > > zacks,
> > > > >> reuters, stockwiz etc etc... and I'm afraid many of the 
sites
> > > > > Brian
> > > > >> has listed).
> > > > >>
> > > > >> So can anyone confirm (Mike perhaps, who has a 
> subscription), 
> > > for
> > > > >> example re. short interest (or any other fundamental 
field): 
> > > will
> > > > > an
> > > > >> AAII subscription truly enable me to pour all past data 
into
> > > > > a .csv
> > > > >> file? No PowerTools or SuperScreens, just good'ol plain 
raw 
> > > data :)
> > > > >>
> > > > >> Stock XYZ short interest:
> > > > >>
> > > > >> Jan '05: ...
> > > > >> Feb '05: ...
> > > > >> .
> > > > >> .
> > > > >> Jun '06: ...
> > > > >>
> > > > >> Thanks for your advice!
> > > > >>
> > > > >> -treliff
> > > > >>
> > > > >> --- In amibroker@xxxxxxxxxxxxxxx, "brian.z123" 
<brian.z123@> 
> > > wrote:
> > > > >> >
> > > > >> > Now you see it, now you don't.
> > > > >> >
> > > > >> > Here it is!
> > > > >> >
> > > > >> > http://betterinvesting.stores.yahoo.net/23303.html
> > > > >> > http://nt3.zacks.com/default.htm
> > > > >> > http://online.barrons.com/public/main
> > > > >> > http://sec.freeedgar.com/
> > > > >> > http://today.reuters.com/news/home.aspx
> > > > >> > http://www.aaii.com/stockinvestor/intro/
> > > > >> > 
> http://www.angelfire.com/in/vestinukgrowthshares/cdrefs.html
> > > > >> > http://www.aspecthuntley.com.au/af/home?xtm-
> > > > > licensee=aspecthuntley
> > > > >> > http://www.capitaliq.com/
> > > > >> > http://www.cbs.curtin.edu.au/index.cfm?
objectId=1B3273A3-
> > 0C32-
> > > > > 2E2E-
> > > > >> > 4B5B799689FF74FC
> > > > >> > http://www.crsp.chicagogsb.edu/
> > > > >> > http://www.csidata.com/
> > > > >> > http://www.exchange-
data.com/products/fundamental_data.aspx
> > > > >> > http://www.fisonline.com/
> > > > >> > http://www.fordequity.com/html/pr_data_usdaily.htm
> > > > >> > http://www.globalfinancialdata.com/index.php3
> > > > >> > http://www.hemscottdata.com/
> > > > >> > http://www.investmenteasy.com/home.aspx
> > > > >> > http://www.it is a capital mistake to theorize in 
advance 
> > of 
> > > the
> > > > >> > facts.Sherlock Holmes
> > > > >> > http://www.market-eye.co.uk/
> > > > >> > http://www.marketdata.sungard.com/
> > > > >> > http://www.msodata.com/
> > > > >> > http://www.perfectinfo.com/product_detail.asp?
productid=4
> > > > >> > http://www.qaisoftware.com/index.html
> > > > >> > http://www.rimes.com/index.xmp
> > > > >> > http://www.securities.com/
> > > > >> > http://www.thomson.com/index.jsp
> > > > >> > http://www.trackdata.com/
> > > > >> > http://www.tradetools.com/web/scripts/default.asp
> > > > >> > http://www.valueline.com/
> > > > >> > http://www.wvb.com/
> > > > >> > http://www.zacks.com/
> > > > >> > http://www2.standardandpoors.com/servlet/Satellite?
> > > > >> > pagename=sp/Page/HomePg&r=1&l=EN&b=10
> > > > >> >
> > > > >> > Some offer full financials, some are summarised 
(financial 
> > or
> > > > > key
> > > > >> > statistics/ratios) and the third possiblity also 
applies.
> > > > >> >
> > > > >> > Brian.
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > --- In amibroker@xxxxxxxxxxxxxxx, "brian.z123" 
> <brian.z123@>
> > > > >> > wrote:
> > > > >> > >
> > > > >> > > Historical fundamental data providers.
> > > > >> > >
> > > > >> > >
> > > > >> > >
> > > > >> > > Here is a list of links to sites that provide 
historical 
> > > fdata
> > > > >> > that
> > > > >> > > may be of interest to AB users when the SQL plug-in is
> > > > > released.
> > > > >> > >
> > > > >> > > Some of them were reviewed by forum members at 
the 'New
> > > > >> > Fundamental
> > > > >> > > Data' and 'Institutional Ownership' topics.
> > > > >> > >
> > > > >> > > Most are USA market biased with some global providers 
> and 
> > a
> > > > >> couple
> > > > >> > > for specific markets (UK and Aus - sorry I don't have 
> any 
> > > for
> > > > >> > other
> > > > >> > > individual markets).
> > > > >> > > They include retail investor providers, investment 
club
> > > > >> providers,
> > > > >> > > academic providers (free to accredited people???) and
> > > > >> > institutional
> > > > >> > > level data (some institutional providers package 
parts of
> > > > > their
> > > > >> > > systems for individual users).
> > > > >> > >
> > > > >> > > The links don't necessarily lead straight to the 
data; 
> > some
> > > > >> > snooping
> > > > >> > > around is required in some cases.
> > > > >> > > It might also contain errors or dead leads.
> > > > >> > >
> > > > >> > > It is not a complete list by any means but includes 
most 
> > of
> > > > > the
> > > > >> > main
> > > > >> > > suspects.
> > > > >> > > Any further leads would be welcomed, including non-
> English
> > > > >> > speaking
> > > > >> > > sites/markets (as long as the post is in English).
> > > > >> > > The criteria for the list is that the database must 
> > include
> > > > > some
> > > > >> > > industry standard fundamental data and be exportable 
in a
> > > > > format
> > > > >> > > that can be used by AB or other similar programs (for 
> > those
> > > > > who
> > > > >> > own
> > > > >> > > more than one software package).
> > > > >> > > For anyone who has info readily available; providers 
home
> > > > >> country,
> > > > >> > > site URL, market, cost, instruments covered, database 
> > > length,
> > > > > EOD
> > > > >> > or
> > > > >> > > weekly data etc, download/update method and 
frequency, 
> > data
> > > > >> > fields,
> > > > >> > > CD available, download or intermediatry software 
needed 
> > and
> > > > >> > download
> > > > >> > > formats available is the type of stuff that provides 
> > useful
> > > > >> > guidance
> > > > >> > > for forum members.
> > > > >> > >
> > > > >> > > I might make a comparison table, and file it at this 
> > site, 
> > > at
> > > > > a
> > > > >> > > later date.
> > > > >> > >
> > > > >> > > Brian.
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Please note that this group is for discussion between 
users 
> > only.
> > > > >
> > > > > To get support from AmiBroker please send an e-mail 
directly 
> to
> > > > > SUPPORT {at} amibroker.com
> > > > >
> > > > > For other support material please check also:
> > > > > http://www.amibroker.com/support.html
> > > > >
> > > > >
> > > > > Yahoo! Groups Links
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>