|
Hello,
EOD vendors (like TC2000, QP2) download quotes for ALL SYMBOLS
at once (if you do update in QP2 for example then data for ALL
stocks
are downloaded, regardless what you are using later). And for
historical data - they come on the CDROM (also for ALL symbols).
That way if you one day decide to see stock "XYZ" it does NOT
need to be downloaded because entire history is in place already (on your hard
disk).
EOD vendors also download corrections and adjustments for
ALL SYMBOLS no matter if you watch them or not.
Real Time vendors NEVER download quotes for symbols that you
do not REQUEST SPECIFICALLY.
So if you decide to see stock "XYZ" it has to be downloaded
from the very beginning. Also if you already had some history
downloaded
earlier but stock had a split it has to be re-downloaded to
get adjusted prices.
As for "automation of RTQ" window such interface is not
available. There are some reasons for that including the fact that
most real time data sources disallow using automated scripts
to switch symbols beyond official limit and download history because it hurts
their servers.
You should understand that bandwidth costs and excesive
downloads costs lots of money. The thinking "I will download off-hours
so I won't hurt them"
is only partially true as bandwidth price is constant and does
NOT depend on hours you use.
Best
regards,
Tomasz Janeczko
amibroker.com
----- Original Message -----
Sent: Saturday, February 04, 2006 9:56
PM
Subject: RE: [amibroker] Retrieve IB data
from more than 100 symbols?
Tomasz,
Thanks. I
appreciate the feedback.
Regarding item 2 (to
take the easier one first), I don’t see any difference between historical EOD
data and historical intraday data. I believe that many (most?) EOD
vendors only download the data that is required. If it’s more than one
day or there have been changes, they just download the data needed.
I.e., they don’t re-download all data every day. That
said…
I suspect that
building that kind of “engine” functionality would require a lot more
work. For you, the cost/benefit ratio may not be good.
L
Regarding item number
1, I perceive that the “Real Time” time needs are different from the
historical intraday data analysis needs. I see two issues with
using the current RTQ window:
1)
Symbol
Limitations And Timing
The RTQ
window is limited to 100 symbols. Frankly, I doubt that I would ever
follow more that 20-40 symbols at a time. It would probably be more like
8-15. OTOH, I can easily see downloading historical intraday
analysis data for several hundred symbols.
Also, as
its name implies, I would use the RTQ window “Real Time” during the US
trading day 9:30 – 4:00 US EST. OTOH, historical intraday data
should be downloaded outside of the US market hours when server load is
low.
2)
Automation and
Scheduling
IMO,
unlike Real Time data, downloading historical intraday needs to be done off
hours, scheduled and automated using a script. Currently, I’ve got
several scripts that are scheduled during the day and evening that download
“LastValue” data from various sources, and then run AB scans and explorations
to process the data for later analysis. The overall process takes
about 70-80 minutes.
The
issue is that I can’t see any way of automating Real Time data
download. I can’t find any exposed objects in the AB Automation
Object Model or any AFL functions that reference the RTQ window (for
adding/deleting symbols and downloading historical
quotes).
Is my analysis
correct? Did I miss something? Any way around these
issues?
Thanks and
regards,
Dan.
From:
amibroker@xxxxxxxxxxxxxxx
[mailto:amibroker@xxxxxxxxxxxxxxx]
On Behalf Of Tomasz Janeczko
Sent: Saturday, February 04, 2006 1:40
AM
To: amibroker@xxxxxxxxxxxxxxx
Subject: Re: [amibroker] Retrieve IB data
from more than 100 symbols?
>1) Ability to provide a list of symbols
(watchlist, text file, whatever) for
download.
This is possible now (via RTQ
window).
>2) Ability to automatically download enough data
to bring it current UP TO a maximum number of days.
>For example, assume that you set the download
filter = 20 days.
> If MSFT had 37 days of 1 minute bars and was
missing the current day, it would download 1 day. If AAPL had no
data, it would download 20 days.
This is how other plugins work (i.e. eSignal, IQfeed,
myTrack) and I will implement this method in IB plugin as
well,
HOWEVER, regardless of data source sometimes it is
necessary to re-download (force backfill) ALL data
since
there may be corrections/adjustments done later and
you may want to get them.
Best regards,
Tomasz
Janeczko
amibroker.com
----- Original Message -----
Sent:
Saturday, February 04, 2006 3:51 AM
Subject: Re:
[amibroker] Retrieve IB data from more than 100
symbols?
Thanks. Good
information.
This is a tough issue because I want to use the
1-minute data for analysis and update the data real-time for trade entry and
exist. It just makes sense to reuse the data. No worries
(hopefully) about database corruption.
I really wish that IB would package the data with a
downloader and charge a reasonable fee for the service. They
have the data already in a database and they provide the API. Ah,
well... I suppose that won't happen because it would be a change in
their business model. That would be
tough.
Regarding your software, what I'd like to
see:
1) Ability to provide a list of symbols (watchlist,
text file, whatever) for download.
2) Ability to automatically download enough data to
bring it current UP TO a maximum number of days. For example, assume
that you set the download filter = 20 days. If MSFT had 37 days of 1
minute bars and was missing the current day, it would download 1
day. If AAPL had no data, it would download 20
days.
-------------- Original message --------------
From: "Tomasz Janeczko"
<amibroker@xxxxxx>
I was not really addressing backfill, but
rather "streaming".
Streaming is limited to 100 simultaneous
symbols.
Backfill is limited to ONE SYMBOL at a time by IB.
If you attempt to access more in parallel it
won't accept backfill request until previous backfill is
complete.
With current plugin the only method to backfill
automatically the list of symbols is to put them into RTQ
window
and use "Backfill all RTQ symbols". This way
plugin will slowly backfill symbols ONE BY
ONE.
Answering second question: yes it will replace
data - for AS MUCH BACK as you have choosen (it may be 1,5,10,15,20,25,30
days).
BTW: I am working on some solution that will
provide methods to get historical data from IB for lots of symbols
easier.
Best regards,
Tomasz
Janeczko
amibroker.com
----- Original Message -----
Sent:
Friday, February 03, 2006 9:50 PM
Subject:
[amibroker] Retrieve IB data from more than 100
symbols?
Hi. I need one-minute
historical data for a 20-day analysis (NOT real-time) on 300-600
symbols. I'd like to use my IB account. In a recent post
(snippet below), Tomasz said that the easiest way of getting IB intraday
data for mutiple symbols is to run an AB scan. The caveat is
that IB is limited to 100 symbols backfill.
So...
- Can I just run multiple
scans, each with a list of 100 symbols?
- For a
specific symbol, does AB append to or replace prior days'
data? For example, today I backfill 20 days data
for MSFT. Tomorrow, I backfill MSFT again. Will
it backfill JUST the incremental data for tomorrow or will it
re-backfill 20 days of data.
-- snip
---------------------------------
The simplest way to do this
for ALL symbols AUTOMATICALLY is to run ONE SCAN (in AA) over all
symbols when you start.
This will cause that plugin
will collect data for ALL symbols included in the scan (if you are using
IB please note that it has limit for 100
symbols).
-- snip
---------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For other support material please check also:
http://www.amibroker.com/support.html
SPONSORED LINKS
YAHOO! GROUPS LINKS
|