[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [amibroker] Re: Speeding up code ???



PureBytes Links

Trading Reference Links



Herman,

That was one of the best descriptions of the various methods that can be used for speed optimizations I have seen.  It really shows the variety of ways to think about RT execution problems and solutions in AFL.  It also highlights how having the low level support in AFL for certain functions can make it easier for those of us who want to do more than a one size fits all solution to trading.  

The beginner should not be concerned about these kinds of things, and many beginners will never progress to serious or complex system development.  For those who do stick with it over the years, the desire and ability to do more with our favorite program grows.  When the time comes to explore and use AB in unique ways that gives us a trading advantage, having more control over low level functions make it a straight forward and rewarding journey.  

However, we do not yet have all those things in place, and our journey may times ends up in a frustrated tangle causing us to wast hundreds of hours, possibly abandon our plans, or jump through system coding hoops to approximate what we want to do.  

I eagerly await each and every new low level AFL feature offered in new versions.  I can almost always use each one to simplify my existing code and make it more reliable in the process -- or implement a long desired ability in my system.  As you have highlighted, it is not just the processing of bars, but also expanding on the user interface that benefits from these methods.

I completely agree with you about the desirability for a more flexible SBR functionality.  It would certainly reduce some of the hoops I am having to jump through now.

My motto is also get the job done in the KISS way!

Best regards,
Dennis

On Jul 3, 2009, at 7:34 AM, Herman wrote:



See capture below...

It may be possible (i haven't tried it) by partitioning code and running them in their own panes with their own unique SetbarsRequired(). 

In my naive opinion, a preferred method would be if AB had a SetBarsRequired() that could be called repeatedly, in effect this would allocated code to different panes without displaying separate panes. For example each Sections_() could have its own SetbarsRequired(); Eventually I think we will see this feature implemented like we saw _Section_()s, TimeFrames(), etc. that can be called repeatedly and control the program environment. 

It would be nice to be able to create/use a virtual pane to isolated and control the environment for afl. 

For personal reasons I like to keep things as easy as possible ;-) hence I prefer to stick to 32bit XP so that I don't run into problems with drivers and other programs i use. The days that I enjoyed installing an OS and learn new features have long gone. At today's level of technology it should be possible to use a computer, like many other sophisticated tools, by just using the GUI. Without know 'how' it works. 

I have found that spending time on optimizing the program design and code is a logical first step that often gives greater benefits than changing hardware. An exception is when you need more than 4GBytes of memory. But proper written Indicator code may not need more than that. 

It is essential to know how AmiBroker processes arrays and how to control bars required. Most of the time the only way to figure this out is to write test programs. 

wrt prioritizing code for speed:

A) Normally code executes on each new quote received. This is the highest order of execution and should be used only for critical stuff, for example polling the last Price to see if it meets you setup requirements.

B) Next you can use the RequestTimedRefresh() to execute code on second (or greater) bounderies. You execute code only when Status("redrawaction") is true.

C) You can execute code dynamically, for example only when the last price (or MA etc) has changed sufficiently to warrant a new execution of a lengthy process.

D) Many function in AT/RT trading only deal with last values. You can often collect (Static variables) a limited number of last values and calculate your own functions. This works for example for MA() and other simple formula that span just few periods. It is faster to calc a Ma(C,5) using numbers than using arrays that use thousands of bars.

E) Results that do not change in the timeframe used should be pre-calculated. They can stored in Static variables, Composites, or files.

F) I have heard people use array-cloning to reduce overhead - i have never used that.

You can try poll RT prices for a watchlist using a simple loop:


Below is a capture of RT afl that polls 498 tickers in Real-Time, calculates values, sorts (OSAKA plugin) them by rank, displays the top n tickers in a bar chart, displays a Button panel for ordering and to display order status. Clicking on a Buy cell places the order, changes its color, and fills it with an order status msg. This idea can be used to trade individual or custom (RT!) indices. The market is now closed so this will execute slower if RT data was coming in, however the 65 Millisecond execution time (IQFeed) shows that a lot can be done with proper programming :-) btw, note that this works because of the minimal use of arrays. 

I am not a professional programmer, I only use afl and simply tinker until it works. My conclusion: afl is VERY FAST!

herman


<int_1.jpg>






Thursday, July 2, 2009, 9:08:04 PM, you wrote:

> Hi Herman,

> I was starting to probe the subject of 'optimal RT performance' in Dennis's
> thread. Trading systems has to be at least as important as desiging them and all trading is RT!

> It is hard for me to stay on the page because a matrix starts to appear, in
> my concsiousness, and I can see the interconnecting threads going in all directions, but I'll try.

> I don't think we have squeezed all of the juice out of the orange by a long
> way but your comments seem to mark out the playing field.

> So far I am just starting to play around with the following ideas:

> - review the OS I am using (any real benefits in using Linux etc)
> - the machines I am using (we have discussed this quite a bit ... the 64 bit story continues?  .. RAM and cache)
> - a dedicated machine (any point in going further than Mike indicates re
> using task manager to prioritize ..... MS comes with a lot of junk I don't use
> ... maybe I will talk to my techie about custom installs for my trading machines ... strip the OS right down
> - Paul's idea of farming out some dirty work to C++ plugins and managing memory allocation there
> - AB develops a brain and slaps me when I am doing less than optimal things with my machine or code
> - managing blocks of AFL to run at different times etc ..... Siddharthas code
> example in Dennis's topic points to one simple way I think.
> - twisting the CPU' arm (in more sophisticated ways than using Windows  Task Manager
> - more to come?

> I don't expect you to answer these questions ... I am just reviewing my
> understanding, of the potential, or the scope, to date.



> Over to your comments:

> Very interesting.

> Have you published anything on the subjects you listed? (if you have covered this at the UKB I will just go there).

>> Anyway, we all have to figure out what works for each of us.

> It sure helps if we pool our philosophies, knowledge and experience first though.

>> you can partition and prioritize RT code, and execute sections of >code on a schedule that doesn't conflict with placing trades. This >allows you to run code code in RT that would lock up any other way. 

> Briefly ... in laypersons terms .. how are you doing that?

>>Using getrtdataforeign() and simple formulas I can scan, in RT, 100 >(easy) to 500 (pushing it) tickers in real time, and display the top >candidates using a gfx chart.

> Very impressive .... so far I have been too scared to even try RT scanning,
> sorting etc .... I find it scary enough in EOD.

> How are you doing that?

>>You may also be able to partition code based on BarsRequired and run >formulas in a separate panes, each with the minimum number of bars >required. Use Static Arrays to share results.

> Any further hints on that?

>>Also, many real time trading routines only need a few prices you can >keep track of yourself - separate these from the arrays processing >pane.

> You seem to be implying that we can use panes to 'partition' code and control
> it's execution. What is the basic execution model that AB uses to manage panes, in relation to each other?




__._,_.___


**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.

TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com

TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/





Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___