[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [amibroker] Re: OT: Yahoo data downloads



PureBytes Links

Trading Reference Links

Yepper - a pain in the ascii for sure.

Just as an FYI - I've written a vb6 app that does most of what you're doing
but with a GUI. I've also got plans to do what you describe for solving the
updating of tickers.

If you're interested we could exchange some info and I'll go ahead and write
that and release the exe and the source code to the AB community. I recall
that you can't release the script as you wrote it for someone else But I'd
be glad to so that everyone can run their own and you won't have to keep
running it and maintaining your site.

I can send you some docs and a sample of the app so you can take a look at,
if you'd like. Just contact me off line if interested.

d 

> -----Original Message-----
> From: amibroker@xxxxxxxxxxxxxxx 
> [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf Of jrswindle2001
> Sent: Thursday, May 24, 2007 4:56 PM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: [amibroker] Re: OT: Yahoo data downloads
> 
> Very true here. I run across the very same problem when I run my
> scripting to do the extracts for the database. My biggest problem is 
> trying to clean up the data. What you say below, is exactly what I
> have ran across too.
> 
> On the new tickers, I have been considering building a utility that
> would read a csv file then verify it with the AB Database, outlining
> the changes that need to be done, instead of just a complete
> overwrite, but haven't gotten to it yet.
> 
> Jim
> 
> --- In amibroker@xxxxxxxxxxxxxxx, "dingo" <dingo@xxx> wrote:
> >
> > Brian, Ronald, et al:
> >  
> > I don't think this particular problem is a good one for 
> scripting. The
> > reasons for saying that are:
> >  
> > 1. In writing some scripting, I found that the data 
> supplied by Yahoo is
> > very "dirty". By that I mean that there are a number of tickers that
> they
> > list on the industry, etc. pages  that are either out of date or
> just plain
> > wrong.  I had to develop a routine to gather all of the 
> tickers and then
> > submit them to another part of Yahoo just to see if they 
> were valid.  In
> > doing so I would get messages that some of them had been changed to
> > something else or they didn't exist.  I ran it last nite 
> and got a new
> > oddball situation: for one ticker it said that it had been changed
> but the
> > new ticker it gave was the exact same as the old one and I hadn't
> programmed
> > for that so I got into an infinite loop in that routine.  Its been
> fixed now
> > but you see the problem.
> >  
> > 2. You'll also get a lot of chaff - tickers for the pink sheets,
> etc. Some
> > people may want them others not - so you've got to provide a way for
> them to
> > cherry pick the ones they want. Doing a GUI in script will 
> add a higher
> > level of complexity to the project.
> >  
> > 3. After all of the tickers have been verified you can still have
> tickers
> > that Yahoo says are "OK" but when you do to download the quotes they
> may not
> > have any.  So, you've got to figure out which one's they 
> are. AQ doesn't
> > tell you - you have to look at all of the files that are 
> downloaded and
> > detect the missing ones.
> >  
> > 4. Then there's the ongoing maintenance problems - new 
> tickers, deleted
> > tickers, changed tickers, etc.
> >  
> > All, in all - for a scripting project (beginning or otherwise) this
> one is
> > very complicated and I think you should learn to crawl first. 
> >  
> > d
> > 
> > 
> >   _____  
> > 
> > From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx]
> On Behalf
> > Of Ronald Davis
> > Sent: Thursday, May 24, 2007 1:25 PM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: Re: [amibroker] Re: OT: Yahoo data downloads
> > 
> > 
> >  
> >  
> > This is just the kind of project I have been hoping to find.
> >  
> > I have no scripting experience at all, but next week, I am starting
> a home
> > study of scripting.  The links below came up instantly on a google
> search,
> > so I am sure that I will have no problem finding enough 
> study material.
> >  
> > If this yahoo data project involves a huge amount of repetitive
> typing of
> > symbols, then I would like to join in on developing the 
> thousands of sub
> > routines that are probably, required.  I learn best when I am
> working on a
> > project that will benefit me. I am willing to devote a substantial
> time to
> > this project.
> >  
> > I know that the time of experienced scripters  is valuable, so would
> ask a
> > question only when I can't find the answer in a google search. 
> >  
> > Maybe there are others wanting to learn at least a little about
> scripting,
> > and would be willing to dovote a substantial amount of time to
> helping.  Ron
> > D
> >
> http://searchwincomputing.techtarget.com/originalContent/0,289
> 142,sid68_gci1
> > 077544,00.html
> >  
> > http://www.asptutorial.info/
> >  
> > 
> http://www.windowsitpro.com/Windows/Article/ArticleID/20376/20376.html
> >  
> >
> http://www.microsoft.com/technet/scriptcenter/topics/beginner/
> firststeps.msp
> > x
> >  
> > http://webdeveloper.com/javascript/javascript_beginners_guide.html
> > ===============================================================
> >  
> > brian_z321 <brian_z321@xxx> wrote:
> > 
> > --- In amibroker@xxxxxxxxx 
> <mailto:amibroker%40yahoogroups.com> ps.com,
> > "dingo" <dingo@> wrote:
> > >
> > > Check your mail.
> > > 
> > > d 
> > 
> > Why did your name spring to mind when I said *kind hearted 
> > scripters*? ;-)
> > 
> > I'll look into it.
> > 
> > Thanks,
> > 
> > Brian.
> > 
> > > > -----Original Message-----
> > > > From: amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
> ps.com 
> > > > [mailto:amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
> ps.com]
> > On Behalf Of brian_z321
> > > > Sent: Wednesday, May 23, 2007 10:23 PM
> > > > To: amibroker@xxxxxxxxx 
> <mailto:amibroker%40yahoogroups.com> ps.com
> > > > Subject: [amibroker] Re: OT: Yahoo data downloads
> > > > 
> > > > If any one is interested let's make the first target an 
> easy one.
> > > > Ee could start with something simple like a script to download 
> > all of 
> > > > the a* or a-aa listings for the NYSE.
> > > > 
> > > > >From there we could progress to getting all of the b* listings 
> > and 
> > > > then try another market for the a*'s.
> > > > 
> > > > Possibly target2 could be to get sectors/inustries for 
> all of the 
> > > > NYSE a* listings.
> > > > 
> > > > With that info I am sure people could work their way up the 
> > ladder.
> > > > 
> > > > No need for anything fancy to start.
> > > > Just get an example list and export it to a .CSV file with a 
> > script.
> > > > 
> > > > Any kind hearted scripters out there?
> > > > 
> > > > Brian.
> > > > 
> > > > --- In amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
> ps.com,
> > "brian_z321" <brian_z321@> 
> > wrote:
> > > > >
> > > > > At the time of writing Yahoo!Finance lists 37,696 
> tickers from 
> > > > approx 
> > > > > 523 exchanges world wide.
> > > > > 
> > > > > They even list the NZ exchange when nobody, except those from 
> > Rugby 
> > > > > Union playing nations, even know that the country exits.
> > > > > 
> > > > > My programming expertise doesn't extend past what I 
> have learnt 
> > > > from 
> > > > > AmiBroker in one single year but I am seriously 
> thinking that I 
> > > > could 
> > > > > manage to learn how to scrape that list for my own use.
> > > > > 
> > > > > Instead of depending on others why not do it for ourselves?
> > > > > 
> > > > > How about a public project, right on this site, where a 
> > community 
> > > > > effort lays out the script, or an example of it, so 
> that we can 
> > all 
> > > > > learn how to do it and manage our own updates and 
> script tuning 
> > in 
> > > > > future.
> > > > > 
> > > > > Pie in the sky?
> > > > > 
> > > > > 
> > > > > Positions vacant:
> > > > > 
> > > > > Project leader, programmers, secretary and working bees.
> > > > > 
> > > > > 
> > > > > Put up now or for ever hold your peace.
> > > > > 
> > > > > 
> > > > > http://biz.yahoo. <http://biz.yahoo.com/i/> com/i/
> > > > > 
> > > > > http://finance. <http://finance.yahoo.com/exchanges>
> > yahoo.com/exchanges
> > > > > 
> > > > > Brian.
> > > > > 
> > > > > P.S. in the unlikely event that we actually do 
> something I will 
> > be 
> > > > > watching from the bleaches; sorry I'm too bsuy elsewhere and 
> > lack 
> > > > the 
> > > > > skill-set for a public effort.
> > > > >
> > > > 
> > > > 
> > > > 
> > > > 
> > > > Please note that this group is for discussion between 
> users only.
> > > > 
> > > > To get support from AmiBroker please send an e-mail directly to 
> > > > SUPPORT {at} amibroker.com
> > > > 
> > > > For NEW RELEASE ANNOUNCEMENTS and other news always 
> check DEVLOG:
> > > > http://www.amibroke <http://www.amibroker.com/devlog/> 
> r.com/devlog/
> > > > 
> > > > For other support material please check also:
> > > > http://www.amibroke <http://www.amibroker.com/support.html>
> > r.com/support.html
> > > > 
> > > > Yahoo! Groups Links
> > > > 
> > > > 
> > > > 
> > > >
> > >
> > 
> > 
> > 
> > 
> > 
> > 
> >   _____  
> > 
> > Be a better Heartthrob. Get
> >
> <http://us.rd.yahoo.com/evt=48255/*http://answers.yahoo.com/di
> r/_ylc=X3oDMTI
> >
> 5MGx2aThyBF9TAzIxMTU1MDAzNTIEX3MDMzk2NTQ1MTAzBHNlYwNCQUJwaWxsY
> XJfTklfMzYwBHN
> > sawNQcm9kdWN0X3F1ZXN0aW9uX3BhZ2U-?link=list&sid=396545433> better
> > relationship answers from someone who knows.
> > Yahoo! Answers - Check it out.
> >
> 
> 
> 
> 
> Please note that this group is for discussion between users only.
> 
> To get support from AmiBroker please send an e-mail directly to 
> SUPPORT {at} amibroker.com
> 
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
> 
> For other support material please check also:
> http://www.amibroker.com/support.html
>  
> Yahoo! Groups Links
> 
> 
> 



Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html
 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/