[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [amibroker] Re: Copying/Limiting Database history



PureBytes Links

Trading Reference Links

As I stated and you reiterated, IO Clones symbols in order to increase speed of optimization. See the comparative run time chart below.

 

It does this in essence by manufacturing and running an AFL to export symbols with limited data history and then importing them under a different easily identifiable symbol.  This is relatively fast and from the perspective of having to match dates totally painless as AB will match the dates when the cloned data is imported.

 

IO does it this way because one might be using IO Servers which can not be expected to have the same completeness of data for the symbol in the same data directory nor even the same symbol in the same data directory nor even the same data directory.

 

 


From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf Of brian_z111
Sent: Friday, May 02, 2008 6:08 AM
To: amibroker@xxxxxxxxxxxxxxx
Subject: [amibroker] Re: Copying/Limiting Database history

 

Tomasz,

I empathise with your position.

When you produce 'customiseable' and complex software the questions
are almost infinite (10000^N where N == newusers).

In a world where transparent exchange of information is the benchmark
isn't getting any easier.

When I said "discussion isn't permanent and info is best provided in
docs or a KB" I put my money where my mouth is.

My new AmiBrokerTradersProject (ATP) is intended to go to the next
level above and beyond the UKB (of course it is up to AB and the
community if they want that or not -if either of those parties don't
want it I will just go to the beach instead).

> It does so for indicators, commentaries, tooltip display and
>interpretations.
> It does NOT do so in AA.

It would have been enough for me if you had only said the above
+ "the exception is when using Pad & Align" (from memory I don't
think that particular special 'bars loaded behaviour' of P&A is
spelled out anywhere in docs - from my side of the screen I can't
guess if all bars are loaded or not, depending on the circumstances).

I am sure a lot of newcomers will be happy to receive your
explanation though.

> I have posted artificial ticker code earlier. If I implement it
>in "easier" way, I am still
> going to "hide" it from "casual user" because of points rised above.

Creating an artificial 'market day' ticker isn't a difficult problem
(rather tedious matching the calendar/market holidays for each market
though but that isn't an AB problem).

I should thing 'advanced' users would be happy to use a function to
over-ride default AA 'load all bars behaviour' and then have to deal
with the consequences (as we do now in Foreign(fixup == 0)).

I don't know about other people but I know how many bars I need at
all times and in all places.

BTW isn't there already some 'problems' to deal with, if using P&A
for optimization truncation, because data with holes will be
automatically fixed, whether the user wants it or not????

Wouldn't we need something like SetBarsRequired(100,0, enabledinAA =
true, fixup = true)????

Anyway, thanks for your answer.

As always you make me think about things.

brian_z

--- In amibroker@xxxxxxxxxps.com, "Tomasz Janeczko" <groups@xxx>
wrote:
>
> Hello,
>
> > I thought AB default was to load a 'few' more bars than required
by
> > AFL
>
> It does so for indicators, commentaries, tooltip display and
interpretations.
>
> It does NOT do so in AA.
>
> Why? It was discussed in the past already.
> I am wondering if any other vendor on this planet gives you inside
out information
> about literally everything. I should probably start saying "because
of technical reasons. period".
> But OK, lets go over it once again.
> There are two main reasons for that:
>
> a) AddToComposite use.
> When you create composites you want them to include all data
consistently,
> not only say one last bar if it happens that you run scan on one
last bar only
>
> b) Trading system consistency and accurracy.
> Your trading system may rely on indicators that have recursive
formulation
> (functions like Cum, EMA, AMA) and others having "unknown" lookback
period
> (ValueWhen). Because of such formulation they use unlimited or
nearly unlimited
> lookback period. Your trading system results may be severely
affected if
> you truncate time series. What does not matter for charts (due to
display limitations
> you can not really see difference of less than 0.5..1% on charts),
may be important
> if you use trading system. The same also applies for exploration.
You probably
> want full accurracy.
>
> If AmiBroker truncated data series in AA by default, we would
simply get
> dozens of e-mails saying "results are inaccurrate/incorrect" simply
because
> most people are unaware of things like recursion. I have seen
people comparing
> absolute values of indicators such as Chaikin Oscillator, without
understading
> its concept of running sum over all loaded bars. Now if AA did not
load all data,
> they would run the same exploration code with different "start"
date and they would
> get different result at the same "end" date.
>
> At support we already spend significant amount of time educating on
such basic things.
>
> For advanced users, who know what they are doing, there is a pad
and align feature
> that allows to use artificial ticker to control on how many bars
your code is run in AA.
> I have posted artificial ticker code earlier. If I implement it
in "easier" way, I am still
> going to "hide" it from "casual user" because of points rised above.
>
> Best regards,
> Tomasz Janeczko
> amibroker.com
> ----- Original Message -----
> From: "brian_z111" <brian_z111@...>
> To: <amibroker@xxxxxxxxxps.com>
> Sent: Friday, May 02, 2008 4:29 AM
> Subject: [amibroker] Re: Copying/Limiting Database history
>
>
> >> Fred and I and probably
> >> others clone symbols to a specified date minus 252 bars to allow
> > long
> >> lookback functions to stabilize. This data truncation has a
> >>PROFOUND
> >> impact on execution times.
> >
> > Can anybody please explain why this speeds up execution.
> >
> > I thought AB default was to load a 'few' more bars than required
by
> > AFL, so if the formula filters by datenum OR range OR barindex
etc
> > why is that slower than using 'Pad and Align' with a 'shortened'
> > symbol.
> >
> > Doesn't AFL only load and process from the filtered date on?
> > If not, what does it do?
> >
> > What does a 'Pad and Align',with a 100 bar reference, do
differently
> > to a barindex filter of 100 days, as far as loading and
processing
> > time goes?
> >
> > I would appreciate it if anyone can clear this up for me.
> >
> > brian_z
> >
> >
> > --- In amibroker@xxxxxxxxxps.com, "bruce1r" <brucer@> wrote:
> >>
> >> Herman -
> >>
> >> First, I agree, the new status functions look wonderful and will
> > allow
> >> some interesting app's.
> >>
> >> In reference to your note about cloning data, I've tried
lobbying
> > TJ
> >> at conferences for 2 years to add this to the Settings. He
> >> acknowledges the gain, so maybe someday soon. Fred and I and
> > probably
> >> others clone symbols to a specified date minus 252 bars to allow
> > long
> >> lookback functions to stabilize. This data truncation has a
> > PROFOUND
> >> impact on execution times. Depending on the amount of AFL code,
and
> >> if you only need a couple years of history, speedups of 3x+ are
not
> >> uncommon. Very important to optimizations.
> >>
> >> There is another way, though, and that is why I thought I'd
write a
> >> quick note. If this method was available through COM
automation, I
> >> would have used it instead of cloning because the overhead is
> > minimal
> >> and the benefits are almost as great.
> >>
> >> Go into settings and set "Pad and align...", and set the
reference
> >> symbol to a ticker that covers the desired date range minus a
number
> >> of bars for functions to stabilize. For example, if you were
using
> >> the QP3 database, you might set it to IEZ, which started in
5/5/2006
> >> to get a 1 year history minus 1 year of bars for stabilization.
You
> >> will get a significant speedup.
> >>
> >> Plus it is simpler than an export/import. Just remember to "un-
set"
> >> it when it isn't needed.
> >>
> >> Bruce R.
> >>
> >
> >
> >
> > ------------------------------------
> >
> > Please note that this group is for discussion between users only.
> >
> > To get support from AmiBroker please send an e-mail directly to
> > SUPPORT {at} amibroker.com
> >
> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > http://www.amibroker.com/devlog/
> >
> > For other support material please check also:
> > http://www.amibroker.com/support.html
> > Yahoo! Groups Links
> >
> >
> >
>

__._,_.___

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html




Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___