|
PureBytes Links
Trading Reference Links
|
Herman,
I too have been battling with maintaining a large database.
I currently use IQFeed and maintain a 500 symbol db at 1 minute resolution. IQFeed currently has about 2 years of intraday data but I keep 10 years of mixed day/intraday data. My DB is about 3.75GB.
Finally these past few weeks I have been able to get the desired performance that I have been looking for. These are the steps I have done to acheive this:
1) I have 8GB of RAM running on Windows XP OS. I created a RamDisk with 4GB of the RAM and copied the entire AB folder + data into this disk. This alone has increased my performance between 4X and 6X. A good utility you can purchase to create a Ram Disk is RAMDisk Plus by Superspeed.
2) I created scripts which Save the DB, unload AB, copy the contents of the Ram Disk to a location on the HDD, load AB and run an exploration with the "wait for backfill" option. This will get all the 500 symbols updated and "online" hence all being updated in realtime. I used the standard Windows XP scheduler to schedule this task to run at night, and gives me a backup which I can reaload in case my machine crashes during market hours. (Note: I was never able to do this with the full 500 symbols without AB crashing at some point until I did the RAM disk thing).
3) Cache size max symbols in AB set to 11, and Max Megabytes set to 850. This causes AB to interact heavily with the HDD, but since the HDD is in fact in RAM (through the RAM Disk) the performance does not degrade at all.
4) With these mods I find I no longer need to run separate DB's, but if you still do, then I found the most reliable is to run AB as a separate instance as a separate user. For this you can first create several logins then create a script which launches AB using the "RUNAS" command.
I am confident that all of these problems are resolved if we could move to AB 64bit, but unfortunately the IQFeed client does not work on a 64bit system, hence it is not an option for the time being.
hope this helps,
eToke
------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|