[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: OT: Yahoo data downloads



PureBytes Links

Trading Reference Links

Very true here. I run across the very same problem when I run my
scripting to do the extracts for the database. My biggest problem is 
trying to clean up the data. What you say below, is exactly what I
have ran across too.

On the new tickers, I have been considering building a utility that
would read a csv file then verify it with the AB Database, outlining
the changes that need to be done, instead of just a complete
overwrite, but haven't gotten to it yet.

Jim

--- In amibroker@xxxxxxxxxxxxxxx, "dingo" <dingo@xxx> wrote:
>
> Brian, Ronald, et al:
>  
> I don't think this particular problem is a good one for scripting. The
> reasons for saying that are:
>  
> 1. In writing some scripting, I found that the data supplied by Yahoo is
> very "dirty". By that I mean that there are a number of tickers that
they
> list on the industry, etc. pages  that are either out of date or
just plain
> wrong.  I had to develop a routine to gather all of the tickers and then
> submit them to another part of Yahoo just to see if they were valid.  In
> doing so I would get messages that some of them had been changed to
> something else or they didn't exist.  I ran it last nite and got a new
> oddball situation: for one ticker it said that it had been changed
but the
> new ticker it gave was the exact same as the old one and I hadn't
programmed
> for that so I got into an infinite loop in that routine.  Its been
fixed now
> but you see the problem.
>  
> 2. You'll also get a lot of chaff - tickers for the pink sheets,
etc. Some
> people may want them others not - so you've got to provide a way for
them to
> cherry pick the ones they want. Doing a GUI in script will add a higher
> level of complexity to the project.
>  
> 3. After all of the tickers have been verified you can still have
tickers
> that Yahoo says are "OK" but when you do to download the quotes they
may not
> have any.  So, you've got to figure out which one's they are. AQ doesn't
> tell you - you have to look at all of the files that are downloaded and
> detect the missing ones.
>  
> 4. Then there's the ongoing maintenance problems - new tickers, deleted
> tickers, changed tickers, etc.
>  
> All, in all - for a scripting project (beginning or otherwise) this
one is
> very complicated and I think you should learn to crawl first. 
>  
> d
> 
> 
>   _____  
> 
> From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx]
On Behalf
> Of Ronald Davis
> Sent: Thursday, May 24, 2007 1:25 PM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: Re: [amibroker] Re: OT: Yahoo data downloads
> 
> 
>  
>  
> This is just the kind of project I have been hoping to find.
>  
> I have no scripting experience at all, but next week, I am starting
a home
> study of scripting.  The links below came up instantly on a google
search,
> so I am sure that I will have no problem finding enough study material.
>  
> If this yahoo data project involves a huge amount of repetitive
typing of
> symbols, then I would like to join in on developing the thousands of sub
> routines that are probably, required.  I learn best when I am
working on a
> project that will benefit me. I am willing to devote a substantial
time to
> this project.
>  
> I know that the time of experienced scripters  is valuable, so would
ask a
> question only when I can't find the answer in a google search. 
>  
> Maybe there are others wanting to learn at least a little about
scripting,
> and would be willing to dovote a substantial amount of time to
helping.  Ron
> D
>
http://searchwincomputing.techtarget.com/originalContent/0,289142,sid68_gci1
> 077544,00.html
>  
> http://www.asptutorial.info/
>  
> http://www.windowsitpro.com/Windows/Article/ArticleID/20376/20376.html
>  
>
http://www.microsoft.com/technet/scriptcenter/topics/beginner/firststeps.msp
> x
>  
> http://webdeveloper.com/javascript/javascript_beginners_guide.html
> ===============================================================
>  
> brian_z321 <brian_z321@xxx> wrote:
> 
> --- In amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com> ps.com,
> "dingo" <dingo@> wrote:
> >
> > Check your mail.
> > 
> > d 
> 
> Why did your name spring to mind when I said *kind hearted 
> scripters*? ;-)
> 
> I'll look into it.
> 
> Thanks,
> 
> Brian.
> 
> > > -----Original Message-----
> > > From: amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
ps.com 
> > > [mailto:amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
ps.com]
> On Behalf Of brian_z321
> > > Sent: Wednesday, May 23, 2007 10:23 PM
> > > To: amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com> ps.com
> > > Subject: [amibroker] Re: OT: Yahoo data downloads
> > > 
> > > If any one is interested let's make the first target an easy one.
> > > Ee could start with something simple like a script to download 
> all of 
> > > the a* or a-aa listings for the NYSE.
> > > 
> > > >From there we could progress to getting all of the b* listings 
> and 
> > > then try another market for the a*'s.
> > > 
> > > Possibly target2 could be to get sectors/inustries for all of the 
> > > NYSE a* listings.
> > > 
> > > With that info I am sure people could work their way up the 
> ladder.
> > > 
> > > No need for anything fancy to start.
> > > Just get an example list and export it to a .CSV file with a 
> script.
> > > 
> > > Any kind hearted scripters out there?
> > > 
> > > Brian.
> > > 
> > > --- In amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
ps.com,
> "brian_z321" <brian_z321@> 
> wrote:
> > > >
> > > > At the time of writing Yahoo!Finance lists 37,696 tickers from 
> > > approx 
> > > > 523 exchanges world wide.
> > > > 
> > > > They even list the NZ exchange when nobody, except those from 
> Rugby 
> > > > Union playing nations, even know that the country exits.
> > > > 
> > > > My programming expertise doesn't extend past what I have learnt 
> > > from 
> > > > AmiBroker in one single year but I am seriously thinking that I 
> > > could 
> > > > manage to learn how to scrape that list for my own use.
> > > > 
> > > > Instead of depending on others why not do it for ourselves?
> > > > 
> > > > How about a public project, right on this site, where a 
> community 
> > > > effort lays out the script, or an example of it, so that we can 
> all 
> > > > learn how to do it and manage our own updates and script tuning 
> in 
> > > > future.
> > > > 
> > > > Pie in the sky?
> > > > 
> > > > 
> > > > Positions vacant:
> > > > 
> > > > Project leader, programmers, secretary and working bees.
> > > > 
> > > > 
> > > > Put up now or for ever hold your peace.
> > > > 
> > > > 
> > > > http://biz.yahoo. <http://biz.yahoo.com/i/> com/i/
> > > > 
> > > > http://finance. <http://finance.yahoo.com/exchanges>
> yahoo.com/exchanges
> > > > 
> > > > Brian.
> > > > 
> > > > P.S. in the unlikely event that we actually do something I will 
> be 
> > > > watching from the bleaches; sorry I'm too bsuy elsewhere and 
> lack 
> > > the 
> > > > skill-set for a public effort.
> > > >
> > > 
> > > 
> > > 
> > > 
> > > Please note that this group is for discussion between users only.
> > > 
> > > To get support from AmiBroker please send an e-mail directly to 
> > > SUPPORT {at} amibroker.com
> > > 
> > > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > > http://www.amibroke <http://www.amibroker.com/devlog/> r.com/devlog/
> > > 
> > > For other support material please check also:
> > > http://www.amibroke <http://www.amibroker.com/support.html>
> r.com/support.html
> > > 
> > > Yahoo! Groups Links
> > > 
> > > 
> > > 
> > >
> >
> 
> 
> 
> 
> 
> 
>   _____  
> 
> Be a better Heartthrob. Get
>
<http://us.rd.yahoo.com/evt=48255/*http://answers.yahoo.com/dir/_ylc=X3oDMTI
>
5MGx2aThyBF9TAzIxMTU1MDAzNTIEX3MDMzk2NTQ1MTAzBHNlYwNCQUJwaWxsYXJfTklfMzYwBHN
> sawNQcm9kdWN0X3F1ZXN0aW9uX3BhZ2U-?link=list&sid=396545433> better
> relationship answers from someone who knows.
> Yahoo! Answers - Check it out.
>




Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html
 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/