|
PureBytes Links
Trading Reference Links
|
The second method has been called data scraping by some folks trying
to create software to enable last second bids on e-bay.
--- In equismetastock@xxxxxxxxxxxxxxx, "L. Carhartt" <lc@xxx> wrote:
>
> I don't know because I don't know what you mean by "basic data
scraping
> techniques". Basically, I parse out a list in one of two ways.
One, I just
> copy the entire web page and paste it into Excel. Saving that as
a .csv
> file and reopening strips out all the formatting and pictures. If
the
> formatting of the list I am after is consistent, I can usually just
delete
> what I don't want and leave only the list. Or Two, I download and
save the
> pages HTML code and find coding consistencies to tell me where the
data is
> that I want and save that to file. I use this second method for
the nightly
> automated update of constituent lists. It took a while to program,
but once
> done, I can update over 140 index/ETF constituent lists in just a
few
> minutes without any input from me.
>
> Usually, the .csv format provided lists on web sites are useless
you are
> doing things manually. Every site seems to have a "better way" and
formats
> are totally different. Hard as it is to believe, its easier just
to use the
> web page HTML.
>
>
> Best,
>
> L. P. Carhartt
>
> MasterDATA
> The Only Source for Index & ETF Composite/Breadth Reports, Charts &
Data
> <http://www.masterdata.com/> www.MasterDATA.com
> <mailto:lc@...> lc@xxx
>
> Important for Subscribers: Please be sure to set up your email
program to
> allow email from the following email addresses:
>
> lc@xxx
> support@xxx
> subscriptions@xxx
>
>
> -----Original Message-----
> From: equismetastock@xxxxxxxxxxxxxxx
[mailto:equismetastock@xxxxxxxxxxxxxxx]
> On Behalf Of rsb_44
> Sent: Wednesday, January 30, 2008 9:50 AM
> To: equismetastock@xxxxxxxxxxxxxxx
> Subject: [EquisMetaStock Group] Re: Inserting Data in the
Downloader Much
> simpler version!!
>
>
>
> PERFECT!
>
> Thanks. I think I can apply this elsewhere. When you convert a web
> page (table) to .csv are you using the basic data scraping
techniques
> from the Source page?
>
> --- In equismetastock@ <mailto:equismetastock%40yahoogroups.com>
> yahoogroups.com, "L. Carhartt" <lc@> wrote:
> >
> > Sorry for the delay. I was making (and then eating) breakfast.
> >
> > I had the web page data you supplied basically converted to.csv
> then started
> > thinking that I knew there had to be a better way. Converting a
> web page to
> > .csv is not particularly difficult, but can be time consuming
when
> done
> > manually. Most of the (index & ETF) constituent lists we gather
> nightly are
> > parsed out of web pages automatically so I am more than familiar
> with the
> > process.
> >
> > Anyway, try this link instead for an already put together .csv
file:
> >
> > http://research.
<http://research.stlouisfed.org/fred2/series/HOUST/>
> stlouisfed.org/fred2/series/HOUST/
> >
> > Under the black headline, click on "Download Data". Scroll down
> the new
> > page and select "Text, comma separated" and then click
on "Download
> Data".
> > Not only is this site THE original source for similar historic
> data, but it
> > is also free. The file provided is easily displayed in MetaStock
> using the
> > MetaStockCSV Plug-in or, if you are a glutton for punishment,
> converted to
> > MetaStock price format files and used that way.
> >
> >
> > Best,
> >
> > L. P. Carhartt
> >
> >
> > -----Original Message-----
> > From: equismetastock@ <mailto:equismetastock%40yahoogroups.com>
> yahoogroups.com
> [mailto:equismetastock@ <mailto:equismetastock%40yahoogroups.com>
> yahoogroups.com]
> > On Behalf Of rsb_44
> > Sent: Wednesday, January 30, 2008 7:45 AM
> > To: equismetastock@ <mailto:equismetastock%40yahoogroups.com>
> yahoogroups.com
> > Subject: [EquisMetaStock Group] Re: Inserting Data in the
> Downloader Much
> > simpler version!!
> >
> >
> >
> > The page given
> >
> > http://www.economag <http://www.economag
<http://www.economagic.com/em->
> ic.com/em-
> cgi/data.exe/feddal/hsm>
> > ic.com/em-cgi/data.exe/feddal/hsm
> >
> > has the data in a table which is inconsistantly formatted.
> > How does one get from the table to a .csv file?
> >
> > BTW Making the Lite version available for free is a real service
to
> > the Excel/Metastock community. Thank you.
> >
> > --- In equismetastock@ <mailto:equismetastock%40yahoogroups.com>
> > yahoogroups.com, "L. Carhartt" <lc@> wrote:
> > >
> > > By the way, I should add, the plug-in will only plot lines and
> > histograms.
> > > For price charts (bar, candlestick charts, etc.) you do need to
> use
> > > MetaStock format price files.
> > >
> > > Best,
> > >
> > > L. P. Carhartt
> > >
> > >
> > > -----Original Message-----
> > > From: equismetastock@ <mailto:equismetastock%40yahoogroups.com>
> > yahoogroups.com
> > [mailto:equismetastock@ <mailto:equismetastock%40yahoogroups.com>
> > yahoogroups.com]
> > > On Behalf Of rsb_44
> > > Sent: Wednesday, January 30, 2008 6:42 AM
> > > To: equismetastock@ <mailto:equismetastock%40yahoogroups.com>
> > yahoogroups.com
> > > Subject: [EquisMetaStock Group] Re: Inserting Data in the
> > Downloader Much
> > > simpler version!!
> > >
> > >
> > >
> > > In Excel Pute the word DATE in A1
> > > in A2 and below put the correct date in mm/dd/yyyy format
> > > In B1 Put the word CLOSE
> > > in B2 and below put the value for the starts. Beware of the
> hidden
> > > characters in the table.
> > > Save the file as an .xls file
> > > Convert the file with the Downloader. (It's help file says how)
> > > Follow the rest of the instructions from the prior message.
> > >
> > > Sorry for the earlier message. It is actually a very simple way
> > with
> > > consistantly formatted data.
> > >
> > > --- In equismetastock@ <mailto:equismetastock%40yahoogroups.com>
> > > yahoogroups.com, "rsb_44" <rsb_44@> wrote:
> > > >
> > > > This isn't a simple task as the data isn't consistantly
> > formatted.
> > > In
> > > > fact copying it by hand doesn't work as some of the data is
> > hidden.
> > > >
> > > > If you know Excel, the broad outline is:
> > > > 1. Copy the data into Excel (The usual click , shift click
copy
> > > paste
> > > > 2. Parse the data in Excel
> > > > 3. Assemble a date
> > > > 4. Convert the date to the format the Downloader accepts
> > > > 5. Carry the actual starts to it's own column.
> > > > 6. Lable the date column DATE and the Data Column CLOSE
> > > > 7. Convert the data using the Downloader (You'll have to
supply
> a
> > > > ticker ID)
> > > > 8. Create an indicator MyClose (It's simply C, but used as an
> > > > indicator.)
> > > > 9. Open the ticker
> > > > 10.Make the plot the same color as the background.
> > > > 11. Apply the myclose indicator use the histogram style.
> > > >
> > > > You wind up with a chart showing housing starts a series of
> > columns
> > > > much like a volumn chart
> > > >
> > > > Is that what you want?
> > > >
> > > >
> > > >
> > > > --- In equismetastock@ <mailto:equismetastock%
40yahoogroups.com>
> > > yahoogroups.com, "Alex Spiroglou"
> > > > <a.spiroglou@> wrote:
> > > > >
> > > > > I would appreciate your help on this please...
> > > > >
> > > > > I would like to chart "Housing Starts" in Metastock.
> > > > > I found the data through Economagic
> > > > > http://www.economag
> > > <http://www.economag
> > <http://www.economag
> <http://www.economagic.com/em-cgi/data.exe/feddal/hsm>
> ic.com/em-cgi/data.exe/feddal/hsm>
> > ic.com/em-cgi/data.exe/feddal/hsm>
> > > ic.com/em-cgi/data.exe/feddal/hsm
> > > > >
> > > > > How could I insert the data in the Downloader ?
> > > > >
> > > > >
> > > > > thank you in advance
> > > > > Alex Spiroglou
> > > > >
> > > > >
> > > > > P.S. I would like to chart data from 2000 and on,
> > > > > so I would not even mind to do it manually
> > > > >
> > > > >
> > > > > [Non-text portions of this message have been removed]
> > > > >
> > > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > [Non-text portions of this message have been removed]
> > >
> >
> >
> >
> >
> >
> >
> >
> > [Non-text portions of this message have been removed]
> >
>
>
>
>
>
>
>
> [Non-text portions of this message have been removed]
>
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/equismetastock/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/equismetastock/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:equismetastock-digest@xxxxxxxxxxxxxxx
mailto:equismetastock-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
equismetastock-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|