|
PureBytes Links
Trading Reference Links
|
Hi,
John was wondering about my strong preference for accessing
data from AB's native format rather than through a plug-in.
First an "executive summary" in a couple paragraphs.
The key to understanding my preferences is to realize I am
working with very large databases containing about 15,000 -
18,000 stocks many of which have 13 or more years of data
history. To increase testing speed, I have a made a
selected watchlist that "just" has 10,500 stocks. The
bigger the database, the greater the speed benefit from
having the data in AB's native format.
Those working with smaller watchlists, may find the
convenience of plugins is worth more than speed increases.
The smaller the database (or watchlist), the more appealing
plugins will be.
Now for more detail on my experience.
Please note that although I have tried out the TC2000 and
QP2 databases and plugins, most of my plugin experience has
been with the MetaStock plugin accessing a fairly large
database of about 15,000 stocks distributed among about 66
MetaStock folders (each holding the standard 250 stocks
which was the MetaStock limit at one time - and maybe still
is).
I have CSI data available to AB in two formats. One is a
csv import that gets stored in AB's native database. The
other is the 66 folder MetaStock type of database which is
accessed in AB via AB's MetaStock plugin.
There are two factors that give the speed advantage to the
native database over the plugin one. The first factor is
the extra steps. To get data from its own database, AB just
calls it. To get data from the MetaStock database, AB has
to ask the MetaStock plug-in and then the MetaStock plug-in
gets the data and then passes that data to AB. Then AB may
also have to so some manipulation of the data to match up
with its internal array structure.
The second, and in my case, the more important factor is I
can get the entire CSI database to fit within a RAM cache
if that database is in AB's native format, but if it is in
the MetaStock format, it will not all fit into the RAM
cache. Thus when AB uses the MetaStock plugin it is always
reading the data from the hard disk for each scan or
optimization run.
I hope this explains why I greatly prefer large databases
being in AB's native format. Of course, if one is working
with a smaller database, or a smaller watchlist within a
database, one might get all the data into RAM cache while
using a plug-in. If so, that would reduce (but not
eliminate) the speed advantage of AB's native database over
a plug-in database.
On the other hand, plug-ins are often much more convenient
to work with than to do massive importing of CSV data into
AB. In the past, it could take the better part of 2.5 hours
to get data out of UA, into CSI format, and finally
imported into AB.
What I like about the new UA export to AB is that the
result puts the data into AB's native format in much less
time (less than 5 minutes for regular updates) and without
any additional steps on my part (except for making sure AB
is open before doing my UA daily download).
The CSI/UA export still seems to have a bug in my
experience. So it is not perfect yet. That bug has been
reported to CSI support others as well as myself. Hopefully
the next UA upgrade (2.7.8) will fix the bugs. The upgrade
may (or may not) be out by the end of the month.
b
--- john gibb <jgibb1@xxxxxxxxxxxxx> wrote:
> Hi,
>
> I am using QuotesPlus data, want to make sure I am
> getting the fastest
> performance possible, and saw this comment from b519b in
> another thread:
>
> "being able to work with CSI data in AB's native
> format is super for
> optimization runs"
>
> Does this imply that:
>
> a) CSI data is not accessed via a plug-in?
> b) Since QuotesPlus data is accessed via a plugin, I
> can't achieve
> performance like that now possible with CSI?
> c) I can't and shouldn't use the "Use only local
> database for this
> symbol" setting with my QP data?
>
> thanks
>
> -john
__________________________________
Do you Yahoo!?
Yahoo! Mail SpamGuard - Read only the mail you want.
http://antispam.yahoo.com/tools
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|