[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [amibroker] Re: Building a 1-minute historical Database



PureBytes Links

Trading Reference Links



Thank you mark, any clarification helps :-)

I wish that some of these settings could be automated, perhaps based on two simple questions:

1) What is the TimeFrame for your new DB?
2) What is the source of your data?

The number of tickers is known as soon as we assign symbols to the DB. All other technical information is already available. It would be nice if it was possible for AB to provide a best-fit setup, and adjust this when conditions change, without the user guessing and calculating blindly. I bet, wrt DB and memory, 95% of the users don't know what they are doing.

thanks for your feedback,
herman

Hi:

I'll take a stab at this - with the understanding that it is pure supposition on my part.


1) What does "File -> Database Settings -> Number of bars" really mean? It doesn't seem to set the "limit" for the data base because I can still  have 250000 bars even if it is set to 50,000. If it refers to "Bars Loaded" why do we have another "Number of Bars to load" in Preferences -> Data ? When is it enforced, only when the DB is created or can it be changed for an old DB? Would this corrupt my DB? Would it reduce the DB  in Size?

* If we assume that the database expands and contracts dynamically, then these settings make sense in terms of optimising application performance.
*  Putting a value in this field allows the application to dynamically allocate the needed space in one go, but still allows the allocated space to grow one bar at a time.
*  The preferences setting referred to in the question is the same concept but instead of referring to disk space it refers to memory space.

2) Is there an absolute limit to the number of bars loaded? What happens when you exceed the DB or Preferences setting, would it corrupt the DB?

* there would be a limit but I would not be worrying about it as you are unlikely to reach it - and if you did the application would warn you.

3) How to set the In-Memory Cache size? Do we have to change this each time we run a different DB? Perhaps someone should design a AmiBroker Setup calculator :-) one of those panels you enter your system/DB parameters in and it tells you what settings to use. Volunteers?

* this would be a global setting that would be applicable to any database, and you would tend to make it as big as possible within the limitation of your own physical memory and operating system.

4) Same question for Max. megabytes...

* the application will use the largest of these two settings.


When changing these settings, when do they take effect?

* the database settings will take effect at database creation time.
* the preferences settings definitely at application startup, possibly at preferences save time.


Thanks
Mark

PS. This could be all be totally wrong, but it is just how I would understand it. It obviously would be much better if a support person could clarify this area.


2009/10/8 Herman <psytek@xxxxxxxx>
 

For DB backups I use dated sub-folders in an AmiBrokerBackup folder. I
don't overwrite anything, I just delete a bunch once in awhile. Disk
space is plentiful and cheap but time is not. When backing up and
restoring AB be aware that you can run into problems restoring if you
backup to a different drive letter.

As someone else mentioned, for processing speed I develop with my large
RT DB set to "Local database", having the data plugin connected really
slows down things.

There are other factors to consider when working with minute DBs. A
clear understanding of what the DB and Preference settings mean, and how
they should be set is essential. After all these years i am still
struggling with some points, perhaps someone can explain in layman's terms:

1) What does "File -> Database Settings -> Number of bars" really mean?
It doesn't seem to set the "limit" for the data base because I can still
have 250000 bars even if it is set to 50,000. If it refers to "Bars
Loaded" why do we have another "Number of Bars to load" in Preferences
-> Data ? When is it enforced, only when the DB is created or can it be
changed for an old DB? Would this corrupt my DB? Would it reduce the DB
in Size?

2) Is there an absolute limit to the number of bars loaded? What happens
when you exceed the DB or Preferences setting, would it corrupt the DB?

3) How to set the In-Memory Cache size? Do we have to change this each
time we run a different DB? Perhaps someone should design a AmiBroker
Setup calculator :-) one of those panels you enter your system/DB
parameters in and it tells you what settings to use. Volunteers?

4) Same question for Max. megabytes...

When changing these settings, when do they take effect?

I know there are ways to calculate memory usage but I create a new DB
almost every day, I wish there was a safe default setting (XP 4GB). If
CacheSize * NumberOfBars * 32 cannot exceed the maxMegabytes the user
should be prevented from doing so. Or is it more complicated that that?

Often when running a portfolio Backtest everything runs fine until at
the very end (after waiting ten minutes) the Backtester gives me an
error. I am pretty sure i am running out of memory, but this is a
technical issue that i shouldn't have to concern myself with.

herman



etoketrader wrote:
> touching on the topic of backups, I fully agree with Herman.
> In my nightly schedule I shut down AB and copy the entire folder as follows:
>
> AB3 -> AB4
> AB2 -> AB3
> AB1 -> AB2
> Live Amimbroker -> AB1
>
> Sounds like an overkill, but it is not the first time I had to go back several copies. Corrupt data is not immediately evident in many cases, especially when dealing with large DB's.
>
> BR,
> eToke
>
>
>
> ------------------------------------

>
> **** IMPORTANT PLEASE READ ****
> This group is for the discussion between users only.
> This is *NOT* technical support channel.
>
> TO GET TECHNICAL SUPPORT send an e-mail directly to
> SUPPORT {at} amibroker.com
>
> TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
> http://www.amibroker.com/feedback/
> (submissions sent via other channels won't be considered)
>
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> Yahoo! Groups Links
>
>
>
>
>
>



__._,_.___


**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.

TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com

TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/





Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___