|
PureBytes Links
Trading Reference Links
|
Hello,
1. Data are stored internally and externally in the base interval you specify in the Database Settings.
If you have chosen 1-minute - it will be 1-minute, if tick - tick.
So whatever you choose - it is used.
2. Yes it uses FIFO.
Best regards,
Tomasz Janeczko
amibroker.com
----- Original Message -----
From: "murthysuresh" <money@xxxxxxxxxxxxxx>
To: <amibroker@xxxxxxxxxxxxxxx>
Sent: Thursday, September 13, 2007 3:30 PM
Subject: [amibroker] Re: Is 500,000 the maximum number of bars allowable in a database?
> Tomasz
> Dollars and cents aside, i would like to understand the criteria that
> is used in the data that would be purged.
> Here is the case that i am trying to understand.
>
> My AB has # o fbars 50000 in base interval 1 minute
> The data feed comign in is a tick level data feed.
>
> Questions are
> 1. Does AB store it as a Tick data internally. The reason is that if
> i want to change to see tick data later on, could i see it for the
> timeframe without it going to the plugin to get the tick data
> 2. AB tells me 50k bars = 104 days with 1562 kb per symbol. So does
> it use FIFO to purge the bar that are >104 days. This means that i
> can backup my database files every 104 days to get some older data if
> i needed.
>
> Seede
> --- In amibroker@xxxxxxxxxxxxxxx, "Tomasz Janeczko" <groups@xxx>
> wrote:
>>
>> That's special price for AmiBroker users :-)
>>
>> Best regards,
>> Tomasz Janeczko
>> amibroker.com
>> ----- Original Message -----
>> From: Don Lindberg
>> To: amibroker@xxxxxxxxxxxxxxx
>> Sent: Thursday, September 13, 2007 6:37 AM
>> Subject: RE: [AmiBroker] Re: Is 500,000 the maximum number of
> bars allowable in a database?
>>
>>
>> Tomasz,
>>
>> Your to cheap. Programming consultants are in the $80 - $100 hr
> range!
>>
>>
>>
>> Donald F Lindberg
>>
>>
>> --------------------------------------------------------------------
> ----------
>>
>> From: amibroker@xxxxxxxxxxxxxxx
> [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf Of eric tao
>> Sent: Wednesday, September 12, 2007 8:55 PM
>> To: amibroker@xxxxxxxxxxxxxxx
>> Subject: [amibroker] Re: Is 500,000 the maximum number of bars
> allowable in a database?
>>
>>
>>
>> $60/hours, Good to know :)
>>
>> --- In amibroker@xxxxxxxxxxxxxxx, "Tomasz Janeczko" <groups@>
> wrote:
>> >
>> > > This isn't even a major change. It just removes the direct
>> > > relationship of per symbol data on disk to the amount of
> symbol data
>> > > Amibroker processes.
>> >
>> > I will tell you something: you have NO idea, absolutely NO
> IDEA, if
>> it is major change or not.
>> > As far as AmiBroker internals are considered your guesses are as
>> accurate as forecasting the value of SP500 in December 2017.
>> >
>> > The one you think isn't major is in fact big (requires in fact
> LOTS
>> of changes) and no-one except 0.5% of AB user base would benefit
>> > from it.
>> >
>> > Are you willing to pay for custom development? $60/hour is the
> rate.
>> > It will take about 10 days of development and testing. Are you
>> willing to pay $4800 to develop it for you?
>> > If so, no problem, I will do that for $4800.
>> >
>> > Best regards,
>> > Tomasz Janeczko
>> > amibroker.com
>> > ----- Original Message -----
>> > From: "scourt2000" <stevehite@>
>> > To: <amibroker@xxxxxxxxxxxxxxx>
>> > Sent: Wednesday, September 12, 2007 4:43 PM
>> > Subject: [amibroker] Re: Is 500,000 the maximum number of bars
>> allowable in a database?
>> >
>> >
>> > >
>> > > Tomasz,
>> > >
>> > > Just because I store large amounts of data in one file (aka, a
>> > > symbol), it doesn't mean that Amibroker has to process all of
> that
>> > > data. Take the example of sqlite, the free database stores
> all of its
>> > > information in one file but users don't process all of that
> data just
>> > > because it's in one physical file on a disk. The stock data in
>> > > Amibroker is a simple database.
>> > >
>> > > Please address this point:
>> > >
>> > > 1. Amibroker should never delete old stock/futures data just
> because
>> > > new data is coming in. All that is needed is for Amibroker to
> limit
>> > > the amount of data it is willing to pull in historically from
> some
>> > > specified start point in the past. Whether that is a certain
> number
>> > > of bars or a specific date range, that's fine. Like I alluded
> to, if
>> > > I have 5 years worth of something like ES tick data as a
> continuous
>> > > contract in one file and I want to backtest (or view) a date
> range in
>> > > that data, then I should have some way in Amibroker of
> specifying a
>> > > start date and end date for the data segment to be pulled in
> for
>> > > processing.
>> > >
>> > > Here's what Amibroker currently 'says': "I'm going to use all
> of the
>> > > data you have for a symbol. If you have too much, then I am
> going to
>> > > blow out the PC's memory. Game over."
>> > >
>> > > Here's what Amibroker should do: If there's too much data for
> a
>> > > symbol, let the user specify a date range for testing/viewing
> or so
>> > > many bars leading up to the present bar for testing/viewing.
> The
>> > > actual physical size of that data on disk remains intact,
> just not all
>> > > available for processing due to memory limitations.
>> > >
>> > > This isn't even a major change. It just removes the direct
>> > > relationship of per symbol data on disk to the amount of
> symbol data
>> > > Amibroker processes.
>> > >
>> > > And like I said previously, there is a workaround for this. I
> have to
>> > > chop up my data into separate, artificial symbol names. Where
> does
>> > > this put the burden of the workload? On me, of course. What is
>> > > software supposed to do? It's supposed to take the burden of
> the
>> > > workload off of the user as much as possible.
>> > >
>> > > Steve
>> > >
>> > >
>> > > --- In amibroker@xxxxxxxxxxxxxxx, "Tomasz Janeczko" <groups@>
> wrote:
>> > >>
>> > >> Hello,
>> > >>
>> > >> Why don't you move to Metastock that has 65000 bar limit ?
>> > >>
>> > >> The truth is that AmiBroker is the most capable in this area
> because
>> > >> it stores way more data than other softwares. Also, by
> changing the
>> > > registry
>> > >> limit you can set it to any value you need. The limit is not
> the
>> > > software but the hardware
>> > >>
>> > >> It is funny when people make statements like yours without
> taking
>> > > calculator and
>> > >> actually checking how much memory you need to store that.
>> > >>
>> > >> 20 million bars - OK, but did you actually count how much
> memory you
>> > > need for that?
>> > >> Each quote is 64 bytes, now 20 million of bars is 1.2GB of
> memory
>> > > per symbol.
>> > >>
>> > >> You may say "but you don't need to load it at once". But
>> > > unfortunatelly you do need that.
>> > >> How you can backtest without loading the data ?
>> > >> And each AFL array in such case would need 4 * 20 million =
> 80 MB of
>> > > RAM.
>> > >> Usually user formulas use hundreds of such arrays.
>> > >>
>> > >> Wise old words are: "You can't eat your cake and have it
> too".
>> > >>
>> > >> If you want more data stored - the price for that is
> increased
>> > > memory requirements
>> > >> and more CPU power needed to process it.
>> > >>
>> > >> Now you can say: you can process one bar at a time - yes you
> can -
>> > > but you need
>> > >> to store intermediate results anyway for next iteration and
> that
>> > > means that memory requirements
>> > >> would not be even one byte less than they are. Plus changing
>> > > execution model to 1-bar at a time will
>> > >> mean (number-of-bars)-times slow down.
>> > >>
>> > >> So you would end up with system that is slow, slow and even
> more
>> slow.
>> > >>
>> > >> Best regards,
>> > >> Tomasz Janeczko
>> > >> amibroker.com
>> > >> ----- Original Message -----
>> > >> From: "scourt2000" <stevehite@>
>> > >> To: <amibroker@xxxxxxxxxxxxxxx>
>> > >> Sent: Wednesday, September 12, 2007 2:00 PM
>> > >> Subject: [amibroker] Re: Is 500,000 the maximum number of
> bars
>> > > allowable in a database?
>> > >>
>> > >>
>> > >> >
>> > >> > And this is my biggest pet peeve about Amibroker:
>> > >> >
>> > >> > If I have the disk space available, Amibroker should never,
>> ever, EVER
>> > >> > just silently blow away my historical data because of some
>> arbitrary
>> > >> > system limit like 500,000 bars or 1,000,000. If I want 20
> million
>> > >> > bars historical and I have the disk space to store it, then
>> fine. If
>> > >> > Amibroker needs far less to function well, then that's
> fine too.
>> > >> >
>> > >> > The solution is NOT to delete old data. The solution is to
>> have a way
>> > >> > for Amibroker to limit the number of past bars (for
> calculation
>> > >> > purposes) or even have me set a beginning and end date in
> an
>> > >> > historical range WITHOUT destroying that historical data
> (i.e., the
>> > >> > present day should not have to be the last bar that I view
> or
>> use for
>> > >> > backtesting).
>> > >> >
>> > >> > Currently, the only solution is to set some registry value
> to
>> go over
>> > >> > 500,000, but that in itself is not solution. Because of
>> Amibroker's
>> > >> > implementation, you'll get your memory stressed out big
> time and,
>> > >> > again, if your new limit gets reached, Amibroker will
> remove your
>> > >> > oldest data to make room for new data.
>> > >> >
>> > >> > No one ever said that you had to look at stock/futures data
>> from the
>> > >> > complete past to today. If I had 20 years of one symbol's
>> data and
>> > >> > that data was far too much to make available at one time
> because of
>> > >> > memory problems, then it should be segmentable without me
> having to
>> > >> > split of the data in artificial symbols.
>> > >> >
>> > >> > Of course there's a workaround (there usually is). You
> have to
>> split
>> > >> > your data up (e.g., tick data) into segments with different
>> artificial
>> > >> > symbol names (e.g., I use eSignal continuous futures
> contracts).
>> > >> >
>> > >> >
>> > >> > --- In amibroker@xxxxxxxxxxxxxxx, Graham <kavemanperth@>
> wrote:
>> > >> >>
>> > >> >> I believe the database limit is set to 500,000 but can be
>> changed in
>> > >> >> the windows registry. TJ has mentioned this in a previous
> post
>> here,
>> > >> >> somewhere sometime.
>> > >> >>
>> > >> >>
>> > >> >> --
>> > >> >> Cheers
>> > >> >> Graham Kav
>> > >> >> AFL Writing Service
>> > >> >> http://www.aflwriting.com
>> > >> >>
>> > >> >>
>> > >> >> On 12/09/2007, Ara Kaloustian <ara1@> wrote:
>> > >> >> >
>> > >> >> >
>> > >> >> > You can have as much as you want, but performance
> degrades.
>> > >> >> >
>> > >> >>
>> > >> >
>> > >> >
>> > >> >
>> > >> >
>> > >> > Please note that this group is for discussion between
> users only.
>> > >> >
>> > >> > To get support from AmiBroker please send an e-mail
> directly to
>> > >> > SUPPORT {at} amibroker.com
>> > >> >
>> > >> > For NEW RELEASE ANNOUNCEMENTS and other news always check
> DEVLOG:
>> > >> > http://www.amibroker.com/devlog/
>> > >> >
>> > >> > For other support material please check also:
>> > >> > http://www.amibroker.com/support.html
>> > >> >
>> > >> > Yahoo! Groups Links
>> > >> >
>> > >> >
>> > >> >
>> > >> >
>> > >> >
>> > >>
>> > >
>> > >
>> > >
>> > >
>> > > Please note that this group is for discussion between users
> only.
>> > >
>> > > To get support from AmiBroker please send an e-mail directly
> to
>> > > SUPPORT {at} amibroker.com
>> > >
>> > > For NEW RELEASE ANNOUNCEMENTS and other news always check
> DEVLOG:
>> > > http://www.amibroker.com/devlog/
>> > >
>> > > For other support material please check also:
>> > > http://www.amibroker.com/support.html
>> > >
>> > > Yahoo! Groups Links
>> > >
>> > >
>> > >
>> > >
>> > >
>> >
>>
>
>
>
>
> Please note that this group is for discussion between users only.
>
> To get support from AmiBroker please send an e-mail directly to
> SUPPORT {at} amibroker.com
>
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> For other support material please check also:
> http://www.amibroker.com/support.html
>
> Yahoo! Groups Links
>
>
>
>
>
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|