|
PureBytes Links
Trading Reference Links
|
Dennis,
<snip>Perhaps, I am the only one who notices or cares about this. If this behavior were fixed so that SBR actually just took the number specified, or had a force mode to cause this to truly set the bars required, even if smaller than the last pass, then I would use more array operations in my AFL, and perhaps gain a speed advantage in the process.<snip>
I am interested in the discussion ... not because I have the need for speed, or have encountered this type of problem so far, but because I am driven to measure the perfection of the ideas behind AB i.e. the design philosophy.
I have touched on one or two of the issues in the past but I can't talk about the subject with the insight I would like to bring to the subject because I am kept out of the play by the fact that AB doesn't set out anywhere to teach us the software architechure .... not anywhere that I have found so far.
My general impression so far is that AB is designed for bulk processing, of bulk arrays == high speed processing.
However, IMO, this means that there has to be a compromise at other times ... if the user is a little more orientated towards other modes i.e. AB is right up the alley of people who want to optimise (three opt engines + the bonus of Fred Tonetti's I/O + the bonus of Howards 'support' of Dakota) ... the latter two nothing to do with Tomasz's decisions.
If your style lies somewhere else then AB's design may not be optimal for that e.g. if you want to import, and run sophisticated analyis of options data, then AB does not offer a 'best of breed' solution (it can be done, using the ODBC/SQL database path but my understanding is that this is for desperados rather than aficiandos).
Another one of AB's design philosophies seems to be to take a generic approach (keep the bulk of the people as happy as possible most of the time while giving advanced users the tools to make their own solutions) ... this is not necessarily 'bad' ... if I were in Tomasz's shoes I would probably find this a 'good', or even essential, approach.
On the other hand, yesterdays cutting edge is todays norm ... sooner or later Tomasz has to make a step up in all core AB features.
One example of this, IMO, is that while 'array' processing is optimal, for optimizing and backtesting, it starts to come into question more in indicator mode (charting).
Once before I identified that this 'bias' was most open to question at the edge ... where whitespace ends and the last bar begins i.e.
- as I understand it (and if I am ignorant on this subject it is not from lack of interest) when a new bar arrives in our chart AB has to reprocess all arrays, for all 'loaded' bars (or all stipulated bars), in our formula.
- QuickAFL and the retired, but resurrected by users, SetBarsRequired is the 'generic' solution to this challenge.
Let's call the area that I am questioning the RT processing edge.
I asked about this, in the forum, before.
Tomasz said something to the effect that array processing, to incorporate newly arrived data, was quicker than looping for one additional bar (if he did not say that, and in fact said something else, he definitely did not acknowledge that there was any value in further consideration of the processing penalty incurred by the default behaviour of 'array processing' in RT charts ...especially very quick ones).
I didn't accept his explanation but I was in not position to reject it either, not having enough information at my disposal.
Given that I have no knowledge, at all, of computing and that I am only considering the issues at a conceptual level:
- what I am questioning is whether or not it is faster, in every single situation, to reprocess all arrays for all bars ... even with QuickAFL on ... or even with SBR, when in indicator mode.
- for example .... if I am calculating an SMA for megabars and new data arrives ... does AB recalculate Sum(allbars)/count(allbars) every refresh (perhaps it does this in the math processor in one step ... sorry but I don't know anything about that stuff but I am willing to learn if the info is placed in an AB context).
If that is the case why wouldn't it be faster to just keep the running sum and keep the running count then re-engineer the math so that the fresh data is included and the new SMA is calced by only performing math processing on two or three datapoints i.e. the running SMA and the new bar.... I think ATC can do this in some way (not sure about that either) but is this the 'best' solution for those who have the need for speed.
I have noted that not all 'indicators' are so amenable to the running total approach i.e. being restated into a twobar solution but it is somewhere to start to consider possible options.
So, while I am not facing the specific issues that you are facing perhaps I am thinking about design issues around that area e.g.
- do we need some other processing mode that we can swing to, besides, array processing, for those who are in fast RT indicator mode (as above)?
- AB seems to be locked into using set modes in set places .. I find this hard to understand at times e.g. if I want to plotShapes * Buy on a chart why do I have to go to AA and run a scan (I am not sure but I think I can do this if I use object orientated code) ... why doesn't AB have AFL like run() that I can use anywhere ... perhaps to only run a certain number of lines of code again without running the complete formula?
- another wild thought ... in RT indicator mode does AB have to wait for a refresh to calc all arrays ... surely it could anticipate the next bar value of some arrays e.g the count and have those arrays preprocessed (Bruce says that Tomasz has optimized all functions ... I don't know exactly what they means of if it is true in all cases or if the above is part of that optimal function processing) i.e in RT indicator mode can, or does, AB precalc the arrays that lend themselves to that and then only reprocess the rest .. can we already 'force' that with what we have .. if not do you see any value in that approach?
Perhaps we can already do this stuff, without any need for further effort by AB ... if so I apologise to Tomasz, and the forum, for not making a bigger effort to read the fine print in the manual.
<snip>When setting parameters, I will
use all 200,000 bars in a backtest equity mode in my indicators. I
switch back and forth many times during the day while developing
systems (even while trading realtime -- in quiet periods).<snip>
It would help me learn a little bit more about AB if you could explain further...
- if you reset parameters then surely you must have to recalc arrays (nothing esle could possibly be done to avoid that?)
- why do you have to use BT equity mode ... and what does switching mean (how do you switch... what do you have to do to achieve that)? ... could anything be done to speed this up or arrive at an alternative (better) method to achieve your ends?
- why recalc all 200,000 bars ... I don't understand why you are not just using the 5-10000 bars you want all the way through?
<snip> In backtest mode, I shut down all nonessential calculations and plots for normal refreshes,<snip>
It seems you have already taken it into your own hands to prioritize what gets reprocessing time and what doesn't ... this must be very tedious ... any suggestions to make it easier to do this type of thing?
<snip> SBR it is better, but it is not so good in one respect. If I just once increase my bars for a quick test, the SBR will never allow me to reduce them back to the lower number again without restarting the chart. I usually restart the chart by applying a sham edit to the main chart AFL. <snip>
What else do you expect AB to do ... off the top of my head I can't see how AB could do anything else but (not with its current priorities)?
Why do you need to do a sham edit just to get a restart of the chart (what is a restart anyway ...is that a refresh of the visible bars only or what?) ... what do you want AB to do in that case ... any suggestions for something better?
<snip> The number of bars used by AFL array operations is a peak detecting function. If anything ever needs more bars (ATC for instance), then you are stuck with that minimum number forever.<snip>
How about ATC(argument1, .... argument2 etc, numberbarsrequired)
OR
SBR;
ATC();
restoreBarsRequired.
It seems that for RT indicator mode we are getting close to asking for the ability to declare variable lengths on a case by case basis ... Tomaz simplified all such things to remove headaches for people like me ... this is against the direction of AB's design philosphy.
Anyway, perahps I am just thinking silly or dangerous thoughts!
OT: psych types
- some comments mainly for you (I think you can use this?)
- my guess is that logic1,intuitive2 (or the inverse) would be the best profile for a trader e.g. I have consciously used AB to strengthen my inverted logic function to facilitate my trading
(it must be painful for a forum of logics to watch an inverted logic learn AB :-) ... progress is very slow, relative to logics who come into the forum.
- do not stereotype the typolgies e.g. a can do (action person) isn't limited to young muscular males etc ... they can be a reserved slim woman (in that case she would suprize many with her quiet achievements ... she would be like a silent machine).
- inverted modes tend to come into expression naturally with age (the only good thing about getting old?)
- exceptional people can have more than two modes functioning well ... that typology would be very happy at the least and very productive in many areas if they so chose (Thorp could have even been one of those ... only first hand contact with him would reveal this),
- once we are familiar with this typology it is the easiest to use for day to day relationships (hiring employees, family relationships etc) ... I can even read the typology of regular posters but won't reveal them for ethical reasons e.g. logic1/feeling2, or the inverse, are very good teachers, actionpeople/felling types are very good in a sports club or social group where they are the glue and get things done without falling out with everyone etc.
--- In amibroker@xxxxxxxxxxxxxxx, Dennis Brown <see3d@xxx> wrote:
>
> Bruce,
>
> Thanks for the comments. I load 200,000 bars in my program, then use
> SetBarsRequired( DesiredHistory, 0 ) to speed up operation to just the
> bars I need at the time for my purposes. When running realtime
> trading, I set my history to 5,000 to 10,000 bars to get all my
> calculations needed (5 to 10 days). When setting parameters, I will
> use all 200,000 bars in a backtest equity mode in my indicators. I
> switch back and forth many times during the day while developing
> systems (even while trading realtime -- in quiet periods).
>
> In real time mode, depending on the system I am running, It can take
> seconds to run each update. I can run up to 8 systems at the same
> time into one combined system for trading (when long and short are
> considered). A second is a long time when you are trading one minute
> bars in a position that may only last 4 minutes on average. I also
> use on chart buttons which really bog down when the refresh rate goes
> over a half second, so I obviously want fast executions.
>
> In backtest mode, I shut down all nonessential calculations and plots
> for normal refreshes, so I can change parameters, and only trigger a
> full calculation cycle with essential plots on command. The quick
> mode is about 5-7 seconds, and the full cycle is about 20-30 seconds.
>
> Now, here is the problem I face. If I allow quick AFL mode, then it
> usually thinks it needs more bars than it does and runs slower. With
> SBR it is better, but it is not so good in one respect. If I just
> once increase my bars for a quick test, the SBR will never allow me to
> reduce them back to the lower number again without restarting the
> chart. I usually restart the chart by applying a sham edit to the
> main chart AFL. My loops will of course only use the number bars
> requested, so they behave well. The number of bars used by AFL array
> operations is a peak detecting function. If anything ever needs more
> bars (ATC for instance), then you are stuck with that minimum number
> forever. That was a reason I could not use ATC operations before, and
> am glad to have the static array variables now.
>
> I have complained about this BAD (for me) behavior many times, but so
> far it seems to have fallen on deaf ears. Perhaps, I am the only one
> who notices or cares about this. If this behavior were fixed so that
> SBR actually just took the number specified, or had a force mode to
> cause this to truly set the bars required, even if smaller than the
> last pass, then I would use more array operations in my AFL, and
> perhaps gain a speed advantage in the process.
>
> BR,
> Dennis
>
>
> On Jun 28, 2009, at 1:00 PM, bruce1r wrote:
>
> >
> >
> > Dennis, all -
> >
> > If you hadn't asked if you weren't missing something, I probably
> > wouldn't post. This is similar to something that Herman and I
> > discussed a week or so back. See those posts, but, IMO, your
> > conclusion about looping is incorrect, and hey, I'll do anything to
> > avoid yard work.
> >
> > You said - "It seems like a waist of time to create an array of the
> > Min/Max over that range for all bars, then just use the last value.
> > I use loops when I only want a single number result related to the
> > Min/Max value over the last n bars only (like the visible bar
> > range). Am I missing an opportunity to approach this in a more
> > efficient way?"
> >
> > Let me try to state the bottom line succinctly -
> >
> > Everyone will most likely eventually run into performance
> > considerations in some application, sooner or later. And, array
> > logic will almost always be significantly faster than looping.
> > Eliminating loops is a good guideline for performance
> > optimization.&nb sp; Plus, if you pay attention to QuickAFL, then
> > speed can be maximized.
> >
> > Tomasz has highly tuned array calculations. Unless you are running
> > them over many 10000's of bars, running over all data will probably
> > out-perform even looping over only visible data - even if you only
> > need the last bar. If you want to consider only visible data and
> > some number of bars, "N", before the first bar that are used to
> > "stabilize" the calculation of indicators, then performance can be
> > maximized. Left on its own QuickAFL will conservatively over-
> > estimate the number of bars required as Tomasz has explain in the KB.
> >
> > If you understand the bar requirements, it is generally better to
> > take control of QuickAFL and limit the all function calculations to
> > "N" bars plus the visible bars. This is done with -
> >
> > SetBarsRequired( N, 0 );
> >
> > The bottom line is that if you do this, array calculations of MACD
> > in your example will outperform looping on visible bars by an order
> > of magnitude or more. And, all other calculations will be sped up
> > considerably also.
> >
> > -- BruceR
> >
> >
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, Dennis Brown <see3d@> wrote:
> > >
> > > Maybe not...
> > >
> > > Through this discussion, the problem has been reduced to aligning
> > the
> > > min/max range of a MACD styleOwnScale plot to another zero line
> > > styleOwnScale plot statement. Therefore, the only way to do this is
> > > to find the min/max values of the MACD function in the visible bars
> > > range and use those numbers in both plot statements, or make the
> > MACD
> > > range symmetrical and then the zero line statement can be any
> > > symmetrical range.
> > >
> > > I am happy with my dual colored plot line at this point. However, to
> > > complete the original idea, I could use a simple loop to find the
> > min/
> > > max numbers of the MACD, but I am now wondering if the re is a
> > "faster"
> > > execution way using array logic instead?
> > >
> > > It seems like a waist of time to create an array of the Min/Max over
> > > that range for all bars, then just use the last value. I use loops
> > > when I only want a single number result related to the Min/Max value
> > > over the last n bars only (like the visible bar range). Am I missing
> > > an opportunity to approach this in a more efficient way?
> > >
> > > BR,
> > > Dennis
> > >
> > >
> > > On Jun 27, 2009, at 9:06 AM, gmorlosky wrote:
> > >
> > > > maybe....
> > > >
> > > > IF ( MACDline == Signalline) // histogram value equals zero
> > > > X = lastvalue(MACDline-Signalline);
> > > > Plot (X,"", colorblack);
> > > >
> > > > --- In amibroker@xxxxxxxxxxxxxxx, Dennis Brown see3d@ wrote:
> > > >>
> > > >> Yes, I have it set to 100%. The absolute values of the indicator
> > > >> will
> > > ; >> vary wildly, depending on the price and volatility of the
> > stock. I
> > > >> plot ES, so the values are quite large at times.
> > > >>
> > > >> BR,
> > > >> Dennis
> > > >>
> > > >> On Jun 26, 2009, at 5:58 PM, monitorit wrote:
> > > >>
> > > >>> If that's all you needed, the color change would be best. But, I
> > > >>> think the solution I suggested would work... you would still
> > plot/
> > > >>> see value very high or low just within a section defined by
> > pane*1/
> > > >>> minvalue [at least it has for me]. BTW, presently what is the
> > upper
> > > >>> and lower bounds of the MACD in relationship to the height of
> > the
> > > >>> pane? Is it 100% of the pane height?
> > > >>> Dan
> > > >>>
> > > >>> --- In amibroker@xxxxxxxxxxxxxxx, Dennis Brown <see3d@> wrote:
> > > >>>>
> > > >>>> ; Dan,
> > > >>>>
> > > >>>> This is what I usually do for indicators when I know the
> > range of
> > > >>>> values. However, MACD is an unbounded indicator, so I don't
> > know
> > > >>>> how
> > > >>>> large the values are. It is only the zero crossings and
> > relative
> > > >>>> peaks that are useful info here.
> > > >>>>
> > > >>>> I just thought there would be some simple shortcut to actually
> > > >>>> scanning the visible bars for the min/max values to use in
> > the plot
> > > >>>> statements. I think I will just switch colors of the MACD
> > from red
> > > >>>> to
> > > >>>> green if it is above or below zero instead of resorting to yet
> > > >>>> another
> > > >>>> loop.
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Dennis
> > > >>>>< br>> >>>>
> > > >>>> On Jun 26, 2009, at 4:40 PM, monitorit wrote:
> > > >>>>
> > > >>>>> Dennis,
> > > >>>>> Hi. You might try this- Place a value in minvalue section but
> > > >>>>> none
> > > >>>>> in maxvalue section [I think maybe a value like 2 equates to
> > 1/2
> > > >>>>> pane height] of your MACD and zero plot statements.
> > > >>>>> Also,styleNoRescale in the zero plot statement. This worked
> > for
> > > >>>>> me
> > > >>>>> when I wanted to plot a histogram and a dot on the top of the
> > > >>>>> histogram. Not sure if it works with a line. If it doesn't,
> > you
> > > >>>>> could try plotohlc with H=max(0,my#) and L=min(0,my#).
> > > >>>>>
> > > >>>>> Dan
> > > >>>>>
> > > >>>>> --- In ami broker@xxxxxxxxxxxxxxx, Dennis Brown <see3d@>
> > wrote:
> > > >>>>>>
> > > >>>>>> Hello,
> > > >>>>>>
> > > >>>>>> This must be simple, but my AFL brain is fried today. I
> > have an
> > > >>>>>> MACD
> > > >>>>>> plot on top of my main price chart, and I want to plot a zero
> > > >>>>>> line
> > > >>>>>> over it. It is plotted as StyleOwnScale of course. I can't
> > use
> > > >>>>>> StyleLeftAxis because I am using it for something else. I
> > > >>>>>> guess I
> > > >>>>>> could do something to figure out where the visible high and
> > low
> > > >>>>>> values
> > > >>>>>> of the MACD are, but it seems like it should be easier than
> > that.
> > > >>>>>>
> > > >>>>>> TIA,
> > > > >>>>> Dennis
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> ------------------------------------
> > > >>>>>
> > > >>>>> **** IMPORTANT PLEASE READ ****
> > > >>>>> This group is for the discussion between users only.
> > > >>>>> This is *NOT* technical support channel.
> > > >>>>>
> > > >>>>> TO GET TECHNICAL SUPPORT send an e-mail directly to
> > > >>>>> SUPPORT {at} amibroker.com
> > > >>>>>
> > > >>>>> TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
> > > >>>>> http://www.amibroker.com/feedback/
> > > >>>>> (submissions sent via other channels won't be considered)
> > > >>>>>
> > > >>>>> For NEW RELEASE ANN OUNCEMENTS and other news always check
> > DEVLOG:
> > > >>>>> http://www.amibroker.com/devlog/
> > > >>>>>
> > > >>>>> Yahoo! Groups Links
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> ------------------------------------
> > > >>>
> > > >>> **** IMPORTANT PLEASE READ ****
> > > >>> This group is for the discussion between users only.
> > > >>> This is *NOT* technical support channel.
> > > >>>
> > > >>> TO GET TECHNICAL SUPPORT send an e-mail directly to
> > > >>> SUPPORT {at} amibroker.com
> > > >>>
> > > >>> TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
> > > >>> http://www.amibroker.com/feedback/
> > > >>> (submissions sent via oth er channels won't be considered)
> > > >>>
> > > >>> For NEW RELEASE ANNOUNCEMENTS and other news always check
> > DEVLOG:
> > > >>> http://www.amibroker.com/devlog/
> > > >>>
> > > >>> Yahoo! Groups Links
> > > >>>
> > > >>>
> > > >>>
> > > >>
> > > >
> > > >
> > > >
> > > >
> > > > ------------------------------------
> > > >
> > > > **** IMPORTANT PLEASE READ ****
> > > > This group is for the discussion between users only.
> > > > This is *NOT* technical support channel.
> > > >
> > > > TO GET TECHNICAL SUPPORT send an e-mail directly to
> > > > SUPPORT {at} amibroker.com
> > > >
> > > > TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
> > > > http://www.amibroker.com/feedback/
> > > > (submissions sent via other channels won't be considered)
> > > >
> > > > For NEW RELEASE ANNOUNCEMENTS a nd other news always check DEVLOG:
> > > > http://www.amibroker.com/devlog/
> > > >
> > > > Yahoo! Groups Links
> > > >
> > > >
> > > >
> > >
> >
> >
> >
>
------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|