[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: OT: Yahoo data downloads



PureBytes Links

Trading Reference Links

Hi D,
I have written some crude VB6 macros for downloading data and 
manipulating in excel (but not from Yahoo). I use a script to run 
the excel macro early in the morning each day. I would be very 
interested to see your approach, if possible. I would certainly 
think it better to use script only to download, then manipilate/ 
clean etc with VB6.
I dont know how to contact you offline if you are ok for me to do 
that. Regrads, Don (Perth Australia).

--- In amibroker@xxxxxxxxxxxxxxx, "dingo" <dingo@xxx> wrote:
>
> I'm confused.. I don't think it'd be too smart for both of us to 
do it, do
> you?
> 
> d 
> 
> > -----Original Message-----
> > From: amibroker@xxxxxxxxxxxxxxx 
> > [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf Of jrswindle2001
> > Sent: Thursday, May 24, 2007 6:34 PM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: [amibroker] Re: OT: Yahoo data downloads
> > 
> > Good idea. I'm looking at rewriting it so it can be released. 
Probably
> > put it in vb.net or c#.net then I can release the source and exe 
to
> > all like you say. I don't have a timetable for it, but I think I 
will
> > put it on the list of things to do.
> > 
> > Jim
> > 
> > --- In amibroker@xxxxxxxxxxxxxxx, "dingo" <dingo@> wrote:
> > >
> > > Yepper - a pain in the ascii for sure.
> > > 
> > > Just as an FYI - I've written a vb6 app that does most of what
> > you're doing
> > > but with a GUI. I've also got plans to do what you describe for
> > solving the
> > > updating of tickers.
> > > 
> > > If you're interested we could exchange some info and I'll go 
ahead
> > and write
> > > that and release the exe and the source code to the AB 
community. I
> > recall
> > > that you can't release the script as you wrote it for someone 
else
> > But I'd
> > > be glad to so that everyone can run their own and you won't 
> > have to keep
> > > running it and maintaining your site.
> > > 
> > > I can send you some docs and a sample of the app so you can 
take a
> > look at,
> > > if you'd like. Just contact me off line if interested.
> > > 
> > > d 
> > > 
> > > > -----Original Message-----
> > > > From: amibroker@xxxxxxxxxxxxxxx 
> > > > [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf Of jrswindle2001
> > > > Sent: Thursday, May 24, 2007 4:56 PM
> > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > Subject: [amibroker] Re: OT: Yahoo data downloads
> > > > 
> > > > Very true here. I run across the very same problem when I 
run my
> > > > scripting to do the extracts for the database. My biggest 
> > problem is 
> > > > trying to clean up the data. What you say below, is exactly 
what I
> > > > have ran across too.
> > > > 
> > > > On the new tickers, I have been considering building a 
> > utility that
> > > > would read a csv file then verify it with the AB 
> > Database, outlining
> > > > the changes that need to be done, instead of just a complete
> > > > overwrite, but haven't gotten to it yet.
> > > > 
> > > > Jim
> > > > 
> > > > --- In amibroker@xxxxxxxxxxxxxxx, "dingo" <dingo@> wrote:
> > > > >
> > > > > Brian, Ronald, et al:
> > > > >  
> > > > > I don't think this particular problem is a good one for 
> > > > scripting. The
> > > > > reasons for saying that are:
> > > > >  
> > > > > 1. In writing some scripting, I found that the data 
> > > > supplied by Yahoo is
> > > > > very "dirty". By that I mean that there are a number of 
> > tickers that
> > > > they
> > > > > list on the industry, etc. pages  that are either out of 
date or
> > > > just plain
> > > > > wrong.  I had to develop a routine to gather all of the 
> > > > tickers and then
> > > > > submit them to another part of Yahoo just to see if they 
> > > > were valid.  In
> > > > > doing so I would get messages that some of them had 
> > been changed to
> > > > > something else or they didn't exist.  I ran it last nite 
> > > > and got a new
> > > > > oddball situation: for one ticker it said that it had 
> > been changed
> > > > but the
> > > > > new ticker it gave was the exact same as the old one 
> > and I hadn't
> > > > programmed
> > > > > for that so I got into an infinite loop in that 
> > routine.  Its been
> > > > fixed now
> > > > > but you see the problem.
> > > > >  
> > > > > 2. You'll also get a lot of chaff - tickers for the pink 
sheets,
> > > > etc. Some
> > > > > people may want them others not - so you've got to 
> > provide a way for
> > > > them to
> > > > > cherry pick the ones they want. Doing a GUI in script will 
> > > > add a higher
> > > > > level of complexity to the project.
> > > > >  
> > > > > 3. After all of the tickers have been verified you can 
> > still have
> > > > tickers
> > > > > that Yahoo says are "OK" but when you do to download 
> > the quotes they
> > > > may not
> > > > > have any.  So, you've got to figure out which one's they 
> > > > are. AQ doesn't
> > > > > tell you - you have to look at all of the files that are 
> > > > downloaded and
> > > > > detect the missing ones.
> > > > >  
> > > > > 4. Then there's the ongoing maintenance problems - new 
> > > > tickers, deleted
> > > > > tickers, changed tickers, etc.
> > > > >  
> > > > > All, in all - for a scripting project (beginning or 
> > otherwise) this
> > > > one is
> > > > > very complicated and I think you should learn to crawl 
first. 
> > > > >  
> > > > > d
> > > > > 
> > > > > 
> > > > >   _____  
> > > > > 
> > > > > From: amibroker@xxxxxxxxxxxxxxx 
> > [mailto:amibroker@xxxxxxxxxxxxxxx]
> > > > On Behalf
> > > > > Of Ronald Davis
> > > > > Sent: Thursday, May 24, 2007 1:25 PM
> > > > > To: amibroker@xxxxxxxxxxxxxxx
> > > > > Subject: Re: [amibroker] Re: OT: Yahoo data downloads
> > > > > 
> > > > > 
> > > > >  
> > > > >  
> > > > > This is just the kind of project I have been hoping to 
find.
> > > > >  
> > > > > I have no scripting experience at all, but next week, I 
> > am starting
> > > > a home
> > > > > study of scripting.  The links below came up instantly 
> > on a google
> > > > search,
> > > > > so I am sure that I will have no problem finding enough 
> > > > study material.
> > > > >  
> > > > > If this yahoo data project involves a huge amount of 
repetitive
> > > > typing of
> > > > > symbols, then I would like to join in on developing the 
> > > > thousands of sub
> > > > > routines that are probably, required.  I learn best when I 
am
> > > > working on a
> > > > > project that will benefit me. I am willing to devote a 
> > substantial
> > > > time to
> > > > > this project.
> > > > >  
> > > > > I know that the time of experienced scripters  is 
> > valuable, so would
> > > > ask a
> > > > > question only when I can't find the answer in a google 
search. 
> > > > >  
> > > > > Maybe there are others wanting to learn at least a little 
about
> > > > scripting,
> > > > > and would be willing to dovote a substantial amount of 
time to
> > > > helping.  Ron
> > > > > D
> > > > >
> > > > 
http://searchwincomputing.techtarget.com/originalContent/0,289
> > > > 142,sid68_gci1
> > > > > 077544,00.html
> > > > >  
> > > > > http://www.asptutorial.info/
> > > > >  
> > > > > 
> > > > 
> > 
http://www.windowsitpro.com/Windows/Article/ArticleID/20376/20376.htm
l
> > > > >  
> > > > >
> > > > 
http://www.microsoft.com/technet/scriptcenter/topics/beginner/
> > > > firststeps.msp
> > > > > x
> > > > >  
> > > > > 
> > 
http://webdeveloper.com/javascript/javascript_beginners_guide.html
> > > > > 
===============================================================
> > > > >  
> > > > > brian_z321 <brian_z321@> wrote:
> > > > > 
> > > > > --- In amibroker@xxxxxxxxx 
> > > > <mailto:amibroker%40yahoogroups.com> ps.com,
> > > > > "dingo" <dingo@> wrote:
> > > > > >
> > > > > > Check your mail.
> > > > > > 
> > > > > > d 
> > > > > 
> > > > > Why did your name spring to mind when I said *kind hearted 
> > > > > scripters*? ;-)
> > > > > 
> > > > > I'll look into it.
> > > > > 
> > > > > Thanks,
> > > > > 
> > > > > Brian.
> > > > > 
> > > > > > > -----Original Message-----
> > > > > > > From: amibroker@xxxxxxxxx 
> > <mailto:amibroker%40yahoogroups.com>
> > > > ps.com 
> > > > > > > [mailto:amibroker@xxxxxxxxx 
> > <mailto:amibroker%40yahoogroups.com>
> > > > ps.com]
> > > > > On Behalf Of brian_z321
> > > > > > > Sent: Wednesday, May 23, 2007 10:23 PM
> > > > > > > To: amibroker@xxxxxxxxx 
> > > > <mailto:amibroker%40yahoogroups.com> ps.com
> > > > > > > Subject: [amibroker] Re: OT: Yahoo data downloads
> > > > > > > 
> > > > > > > If any one is interested let's make the first target 
an 
> > > > easy one.
> > > > > > > Ee could start with something simple like a script 
> > to download 
> > > > > all of 
> > > > > > > the a* or a-aa listings for the NYSE.
> > > > > > > 
> > > > > > > >From there we could progress to getting all of the 
> > b* listings 
> > > > > and 
> > > > > > > then try another market for the a*'s.
> > > > > > > 
> > > > > > > Possibly target2 could be to get sectors/inustries for 
> > > > all of the 
> > > > > > > NYSE a* listings.
> > > > > > > 
> > > > > > > With that info I am sure people could work their way 
up the 
> > > > > ladder.
> > > > > > > 
> > > > > > > No need for anything fancy to start.
> > > > > > > Just get an example list and export it to a .CSV 
> > file with a 
> > > > > script.
> > > > > > > 
> > > > > > > Any kind hearted scripters out there?
> > > > > > > 
> > > > > > > Brian.
> > > > > > > 
> > > > > > > --- In amibroker@xxxxxxxxx 
> > <mailto:amibroker%40yahoogroups.com>
> > > > ps.com,
> > > > > "brian_z321" <brian_z321@> 
> > > > > wrote:
> > > > > > > >
> > > > > > > > At the time of writing Yahoo!Finance lists 37,696 
> > > > tickers from 
> > > > > > > approx 
> > > > > > > > 523 exchanges world wide.
> > > > > > > > 
> > > > > > > > They even list the NZ exchange when nobody, 
> > except those from 
> > > > > Rugby 
> > > > > > > > Union playing nations, even know that the country 
exits.
> > > > > > > > 
> > > > > > > > My programming expertise doesn't extend past what I 
> > > > have learnt 
> > > > > > > from 
> > > > > > > > AmiBroker in one single year but I am seriously 
> > > > thinking that I 
> > > > > > > could 
> > > > > > > > manage to learn how to scrape that list for my own 
use.
> > > > > > > > 
> > > > > > > > Instead of depending on others why not do it for 
> > ourselves?
> > > > > > > > 
> > > > > > > > How about a public project, right on this site, 
where a 
> > > > > community 
> > > > > > > > effort lays out the script, or an example of it, so 
> > > > that we can 
> > > > > all 
> > > > > > > > learn how to do it and manage our own updates and 
> > > > script tuning 
> > > > > in 
> > > > > > > > future.
> > > > > > > > 
> > > > > > > > Pie in the sky?
> > > > > > > > 
> > > > > > > > 
> > > > > > > > Positions vacant:
> > > > > > > > 
> > > > > > > > Project leader, programmers, secretary and working 
bees.
> > > > > > > > 
> > > > > > > > 
> > > > > > > > Put up now or for ever hold your peace.
> > > > > > > > 
> > > > > > > > 
> > > > > > > > http://biz.yahoo. <http://biz.yahoo.com/i/> com/i/
> > > > > > > > 
> > > > > > > > http://finance. <http://finance.yahoo.com/exchanges>
> > > > > yahoo.com/exchanges
> > > > > > > > 
> > > > > > > > Brian.
> > > > > > > > 
> > > > > > > > P.S. in the unlikely event that we actually do 
> > > > something I will 
> > > > > be 
> > > > > > > > watching from the bleaches; sorry I'm too bsuy 
> > elsewhere and 
> > > > > lack 
> > > > > > > the 
> > > > > > > > skill-set for a public effort.
> > > > > > > >
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > Please note that this group is for discussion between 
> > > > users only.
> > > > > > > 
> > > > > > > To get support from AmiBroker please send an e-mail 
> > directly to 
> > > > > > > SUPPORT {at} amibroker.com
> > > > > > > 
> > > > > > > For NEW RELEASE ANNOUNCEMENTS and other news always 
> > > > check DEVLOG:
> > > > > > > http://www.amibroke <http://www.amibroker.com/devlog/> 
> > > > r.com/devlog/
> > > > > > > 
> > > > > > > For other support material please check also:
> > > > > > > http://www.amibroke 
<http://www.amibroker.com/support.html>
> > > > > r.com/support.html
> > > > > > > 
> > > > > > > Yahoo! Groups Links
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > >
> > > > > >
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > >   _____  
> > > > > 
> > > > > Be a better Heartthrob. Get
> > > > >
> > > > 
<http://us.rd.yahoo.com/evt=48255/*http://answers.yahoo.com/di
> > > > r/_ylc=X3oDMTI
> > > > >
> > > > 
5MGx2aThyBF9TAzIxMTU1MDAzNTIEX3MDMzk2NTQ1MTAzBHNlYwNCQUJwaWxsY
> > > > XJfTklfMzYwBHN
> > > > > 
> > sawNQcm9kdWN0X3F1ZXN0aW9uX3BhZ2U-?link=list&sid=396545433> better
> > > > > relationship answers from someone who knows.
> > > > > Yahoo! Answers - Check it out.
> > > > >
> > > > 
> > > > 
> > > > 
> > > > 
> > > > Please note that this group is for discussion between users 
only.
> > > > 
> > > > To get support from AmiBroker please send an e-mail directly 
to 
> > > > SUPPORT {at} amibroker.com
> > > > 
> > > > For NEW RELEASE ANNOUNCEMENTS and other news always check 
DEVLOG:
> > > > http://www.amibroker.com/devlog/
> > > > 
> > > > For other support material please check also:
> > > > http://www.amibroker.com/support.html
> > > >  
> > > > Yahoo! Groups Links
> > > > 
> > > > 
> > > >
> > >
> > 
> > 
> > 
> > 
> > Please note that this group is for discussion between users only.
> > 
> > To get support from AmiBroker please send an e-mail directly to 
> > SUPPORT {at} amibroker.com
> > 
> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > http://www.amibroker.com/devlog/
> > 
> > For other support material please check also:
> > http://www.amibroker.com/support.html
> >  
> > Yahoo! Groups Links
> > 
> > 
> >
>




Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html
 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/