|
PureBytes Links
Trading Reference Links
|
A technique I've been using to deal with large intraday historical DB's is to store the data in a SQL database, use the ODBC plugin (with some custom mods) and use a SQL stored procedure to fetch the data and deliver the bars to AB. Then you can control how much data AB "sees" (has access to) by changing a variable in some SQL table that your your stored procedure uses indicating the number of bars/days you want to have delivered. The system I created for doing all of this is too complicated, proprietary, and valuable to give out in code, but that's the general idea....
You need an adjustable layer of abstraction between AB and the data source, and that's how I handle it. In my case, this layer of abstraction, handled by the stored procedure in the SQL database, is also capable of delivering the desired size of the bars... I have SQL bar building code that creates different tables for different bar intervals from the 1 minute data, and rather than leaving it up to AB to assemble the 1m bars into, for instance, 5 minute bars (which burdens AB significantly for a large 1m database), I use my prebuilt QuotesMin5 SQL table. That simple thing allows me to work with 5 times the history with 1/5th the memory/CPU requirements, as long as I don't need to look into bars smaller than 5 minutes.
That kind of thing.... So maybe that sparks some ideas for y'all.
Certainly not an easy solution for the masses.... but the determined few will always find a way.
-Longstrangest
One man's dream for AB: If only AB itself would use multithreading/multiprocessing to assemble bars on time intervals different than the database (or using multithreading/multiprocessing for *anything* for that matter).... Then maybe the other 3 cores on my quad core CPU would get some use. Still scratching my head and wondering why AB maxes out one core constantly, and never uses the other processors, in this day and age where the only significant performance improvements we've seen in CPU power has been achieved with multiple CPU cores on a single die. Until AB becomes multiprocessing-aware, I'll be forced to write code that duplicates AB functionality, cave-man style, like bar building, that I can run in my own separate thread. IMO, there's no better platform than AB.... but if there's one single monstrous performance improvement AB can make, it's taking advantage of multiprocessing.
<drums fingers on desk, taps foot>
Dream on.
------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
amibroker-digest@xxxxxxxxxxxxxxx
amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|