[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [amibroker] Re: Copying/Limiting Database history



PureBytes Links

Trading Reference Links

Very nice work...


herman


For tips on developing Real-Time Auto-Trading systems visit:

http://www.amibroker.org/userkb/


Friday, May 2, 2008, 7:44:00 AM, you wrote:


>

As I stated and you reiterated, IO Clones symbols in order to increase speed of optimization. See the comparative run time chart below.

 

It does this in essence by manufacturing and running an AFL to export symbols with limited data history and then importing them under a different easily identifiable symbol.  This is relatively fast and from the perspective of having to match dates totally painless as AB will match the dates when the cloned data is imported.

 

IO does it this way because one might be using IO Servers which can not be expected to have the same completeness of data for the symbol in the same data directory nor even the same symbol in the same data directory nor even the same data directory.

 

 


From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf Of brian_z111

Sent: Friday, May 02, 2008 6:08 AM

To: amibroker@xxxxxxxxxxxxxxx

Subject: [amibroker] Re: Copying/Limiting Database history

 

Tomasz,


I empathise with your position.


When you produce 'customiseable' and complex software the questions 

are almost infinite (10000^N where N == newusers).


In a world where transparent exchange of information is the benchmark 

isn't getting any easier.


When I said "discussion isn't permanent and info is best provided in 

docs or a KB" I put my money where my mouth is.


My new AmiBrokerTradersProject (ATP) is intended to go to the next 

level above and beyond the UKB (of course it is up to AB and the 

community if they want that or not -if either of those parties don't 

want it I will just go to the beach instead).


> It does so for indicators, commentaries, tooltip display and 

>interpretations. 

> It does NOT do so in AA. 


It would have been enough for me if you had only said the above 

+ "the exception is when using Pad & Align" (from memory I don't 

think that particular special 'bars loaded behaviour' of P&A is 

spelled out anywhere in docs - from my side of the screen I can't 

guess if all bars are loaded or not, depending on the circumstances).


I am sure a lot of newcomers will be happy to receive your 

explanation though.


> I have posted artificial ticker code earlier. If I implement it 

>in "easier" way, I am still

> going to "hide" it from "casual user" because of points rised above.


Creating an artificial 'market day' ticker isn't a difficult problem 

(rather tedious matching the calendar/market holidays for each market 

though but that isn't an AB problem).


I should thing 'advanced' users would be happy to use a function to 

over-ride default AA 'load all bars behaviour' and then have to deal 

with the consequences (as we do now in Foreign(fixup == 0)).


I don't know about other people but I know how many bars I need at 

all times and in all places.


BTW isn't there already some 'problems' to deal with, if using P&A 

for optimization truncation, because data with holes will be 

automatically fixed, whether the user wants it or not????


Wouldn't we need something like SetBarsRequired(100,0, enabledinAA = 

true, fixup = true)????


Anyway, thanks for your answer.


As always you make me think about things.


brian_z


--- In amibroker@xxxxxxxxxps.com, "Tomasz Janeczko" <groups@xxx> 

wrote:

>

> Hello,

> > I thought AB default was to load a 'few' more bars than required 

by 

> > AFL

> It does so for indicators, commentaries, tooltip display and 

interpretations.

> It does NOT do so in AA. 

> Why? It was discussed in the past already. 

> I am wondering if any other vendor on this planet gives you inside 

out information

> about literally everything. I should probably start saying "because 

of technical reasons. period".

> But OK, lets go over it once again.

> There are two main reasons for that:

> a) AddToComposite use. 

> When you create composites you want them to include all data 

consistently,

> not only say one last bar if it happens that you run scan on one 

last bar only

> b) Trading system consistency and accurracy. 

> Your trading system may rely on indicators that have recursive 

formulation

> (functions like Cum, EMA, AMA) and others having "unknown" lookback 

period

> (ValueWhen). Because of such formulation they use unlimited or 

nearly unlimited

> lookback period. Your trading system results may be severely 

affected if 

> you truncate time series. What does not matter for charts (due to 

display limitations

> you can not really see difference of less than 0.5..1% on charts), 

may be important

> if you use trading system. The same also applies for exploration. 

You probably

> want full accurracy.

> If AmiBroker truncated data series in AA by default, we would 

simply get

> dozens of e-mails saying "results are inaccurrate/incorrect" simply 

because

> most people are unaware of things like recursion. I have seen 

people comparing

> absolute values of indicators such as Chaikin Oscillator, without 

understading

> its concept of running sum over all loaded bars. Now if AA did not 

load all data,

> they would run the same exploration code with different "start" 

date and they would

> get different result at the same "end" date.

> At support we already spend significant amount of time educating on 

such basic things.

> For advanced users, who know what they are doing, there is a pad 

and align feature

> that allows to use artificial ticker to control on how many bars 

your code is run in AA.

> I have posted artificial ticker code earlier. If I implement it 

in "easier" way, I am still

> going to "hide" it from "casual user" because of points rised above.

> Best regards,

> Tomasz Janeczko

> amibroker.com

> ----- Original Message ----- 

> From: "brian_z111" <brian_z111@xxx>

> To: <amibroker@xxxxxxxxxps.com>

> Sent: Friday, May 02, 2008 4:29 AM

> Subject: [amibroker] Re: Copying/Limiting Database history

> >> Fred and I and probably

> >> others clone symbols to a specified date minus 252 bars to allow 

> > long

> >> lookback functions to stabilize. This data truncation has a 

> >>PROFOUND

> >> impact on execution times. 

> > 

> > Can anybody please explain why this speeds up execution.

> > 

> > I thought AB default was to load a 'few' more bars than required 

by 

> > AFL, so if the formula filters by datenum OR range OR barindex 

etc 

> > why is that slower than using 'Pad and Align' with a 'shortened' 

> > symbol.

> > 

> > Doesn't AFL only load and process from the filtered date on?

> > If not, what does it do?

> > 

> > What does a 'Pad and Align',with a 100 bar reference, do 

differently 

> > to a barindex filter of 100 days, as far as loading and 

processing 

> > time goes?

> > 

> > I would appreciate it if anyone can clear this up for me.

> > 

> > brian_z

> > 

> > 

> > --- In amibroker@xxxxxxxxxps.com, "bruce1r" <brucer@> wrote:

> >>

> >> Herman -

> >> 

> >> First, I agree, the new status functions look wonderful and will 

> > allow

> >> some interesting app's.

> >> 

> >> In reference to your note about cloning data, I've tried 

lobbying 

> > TJ

> >> at conferences for 2 years to add this to the Settings. He

> >> acknowledges the gain, so maybe someday soon. Fred and I and 

> > probably

> >> others clone symbols to a specified date minus 252 bars to allow 

> > long

> >> lookback functions to stabilize. This data truncation has a 

> > PROFOUND

> >> impact on execution times. Depending on the amount of AFL code, 

and

> >> if you only need a couple years of history, speedups of 3x+ are 

not

> >> uncommon. Very important to optimizations.

> >> 

> >> There is another way, though, and that is why I thought I'd 

write a

> >> quick note. If this method was available through COM 

automation, I

> >> would have used it instead of cloning because the overhead is 

> > minimal

> >> and the benefits are almost as great.

> >> 

> >> Go into settings and set "Pad and align...", and set the 

reference

> >> symbol to a ticker that covers the desired date range minus a 

number

> >> of bars for functions to stabilize. For example, if you were 

using

> >> the QP3 database, you might set it to IEZ, which started in 

5/5/2006

> >> to get a 1 year history minus 1 year of bars for stabilization. 

You

> >> will get a significant speedup.

> >> 

> >> Plus it is simpler than an export/import. Just remember to "un-

set"

> >> it when it isn't needed.

> >> 

> >> Bruce R.

> >>

> > 

> > 

> > 

> > ------------------------------------

> > 

> > Please note that this group is for discussion between users only.

> > 

> > To get support from AmiBroker please send an e-mail directly to 

> > SUPPORT {at} amibroker.com

> > 

> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:

> > http://www.amibroker.com/devlog/

> > 

> > For other support material please check also:

> > http://www.amibroker.com/support.html

> > Yahoo! Groups Links

> > 

> > 

> >

>

 

__._,_.___

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html




Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___