|
PureBytes Links
Trading Reference Links
|
Hi Thomas,
To the article:
GDI and Windows graphics are not my special area. However, this article
talks about scaling MANY applications and contention among them.
On my real time trading workstation I use only AmiBroker. Only required apps
are deployed there. Only AB's window is open. IBController and TWS is
minimized. DebugView may run eventually minimized. So I guess GDI locks are
not my issue. Correct me if I'm wrong.
I also read this : "Contention due to the global exclusive lock is avoided
by implementing a number of fine-grained locks which are not exclusive but
aid parallelism. The increased number of fine-grained locks adds a small
overhead for scenarios where only a single application is rendering at a
time."
So there might be some improvement from the more efficient GDI subsystem
(memory handling, new multi monitor handling, new driver, etc.), and some
losses from these new overheads.
So my instincs say that the picture is not so clear! There would be no major
breakthrough in case of a "dedicated trading machine". Only users, how use
other apps concurrently will see some improvements. But those (I believe)
typically do not use AB for real time trading.
To the requests:
You are right. We have only "believes" about AmiBroker internals, while you
have the code and the possibility to profile it.
You are also in charge to allocate time and resource to implement any new
features and to rank them what to do first.
What you do not know for sure is how others use AmiBroker!
So, please listen to YOUR CUSTOMERS... We pay you anyway...
You may still decide that requested feature affects only a few users and you
don't implement it or you do it in a more meaningful way.
I personally try to trade only 1-4 equities intraday. All traded equities
have a single chart with a single indicator pane in it.
This indicator pane generates the trading signals (indicators, entry
signals), trade management (trailing, cancelling outstanding orders, etc.)
and it does trading (state management, order status, etc.) as well.
It also let me controll trading (stop, start, etc.). It displays the actual
status of the trading/orders as well. Everything is automatic, so we use
periodic checks only.
What is on the display (charts) we trade it! I do not want to set up
watchlists and bother AA window, etc.
We try to do it in time shift. So I do not want to tell the next guy what
watchlists I changed...
We try to avoid all human errors. So charts are our only UI interface!
My experience (believe) is that script parsing (or some kind of pre running
setup work) and trade/state management logic (IB Controller calls and
surrounding AFL state management and calculation loops) takes most of the
chart refresh cycle. My indicators are not heavy! I have basic OHLC chart
with an EMA and 2 other lines. That's it! I guess it does not require a lot
of GDI calls.
Still we experience slow execution because of heavy logic. It is because AB
behaves like a single threaded application.
My GUESS is that all chart refreshes are queued and than all calculation and
drawing are done by the app main thread sequentially (The only one allowed
to access UI/GUI).
The consequesnce is known by all. Either all charts work fine or none of
them.
This is why I asked for some change to make AB more efficient. We are not
interested in all tick. So I'm not asking for repainting the bars and
indicators at each tick. But I want to have trading logic run at every
seconds. And I want to have the correct controlling information on the
panel.
So if calculations could be executed on any thread and plot/paint/Gfx
function calls queued, the AFL code execution could be much faster and more
frequent and more chart code could be processed. Charts could be independent
of each other. If one chart has slow code the others could still run
normally. If we could controll when to do repaint of the chart from queued
painting data we still have corrent info on the panel.
I guess Dennis has some similar problem.
Regards,
Y
--------------------------------------------------
From: "Tomasz Janeczko" <groups@xxxxxxxxxxxxx>
Sent: Sunday, July 12, 2009 10:41 PM
To: <amibroker@xxxxxxxxxxxxxxx>
Subject: Re: [amibroker] Per Chart Refresh Rates
> Hello,
>
> There is no sense in doing indicator calculation when this calculation
> does not lead to actual rendering. That would be waste of CPU.
> The purpose of doing indicator calculation is to actually display it
> (refresh it).
> Indicators formulas are here to display something.
>
> If you want code that does not display anything, you should run it as
> automatic-analysis (scan/exploration) code.
>
> Also AFL formula execution is often much faster than final GDI output,
> therefore even if AFL formula was executed in parallel, it would still
> face GDI contention because of Windows GDI system-wide lock.
>
> Only on Windows 7 this system-wide GDI lock is removed and only
> there you could see graphic performance scaling
>
> Again, read the entire article:
> http://blogs.msdn.com/e7/archive/2009/04/25/engineering-windows-7-for-graphics-performance.aspx
>
> Especially figure 4 GDI Concurrency and Scalability
> - as you can see in any pre-Win7 systems, GDI does not scale *at all*
> with adding threads.
>
> I can ensure you that I have actually *timed* many scenarios
> and what I say is based on actual measurement and not on somebody's
> "belief".
> That was one of the reasons I did not use GDI+ ("improvement" suggested
> by somebody on this list)
> because real-world test revealed that it is 6 times slower than normal
> GDI.
> Microsoft admitted that by the way in their recent demonstration on PDC
> (prof. dev. conference).
>
> So, all decisions regarding development of AmiBroker are not based on
> beliefs but on
> hard code profiling evidence.
>
> Best regards,
> Tomasz Janeczko
> amibroker.com
> ----- Original Message -----
> From: "Yofa" <jtoth100@xxxxxxxxxxx>
> To: <amibroker@xxxxxxxxxxxxxxx>
> Sent: Sunday, July 12, 2009 9:47 PM
> Subject: Re: [amibroker] Per Chart Refresh Rates
>
>
>> Hi Thomas,
>>
>>> Tomasz wrote in the "Data And PlugIn Speed" thread
>>>I may add that the concept of independent rendering from multiple threads
>>>although attractive from first look, it inevitably hits the wall of GDI
>>>which in all current versions of Windows has
>>> single system-wide exclusive global lock, which means that only ONE
>>> thread
>>> can actually render at one time. This means that adding threads does not
>>> give you better performance.
>>
>>>The truth is that all current releases of Windows are not particularly
>>>suited for multi-core/multi-CPU
>>>but good news is that Microsoft apparently have given these limitations
>>>some thought and
>>>many of them are removed in Windows 7.
>>
>> That is true for rendering only! (Apps main thread is allowed to write
>> the
>> GDI device. So rendering is limited to a single thread.)
>>
>> But indicator calculation (and trading logic) could get executed by any
>> threads. Am I right? So doing the calculation on a background thread than
>> doing chart painting with the "main" thread would increase processor
>> usage
>> and increase chart refresh reate? (I guess we all have dual and quad core
>> CPUs...)
>>
>> How about separating "calculation refresh rate" and "chart refresh rate"?
>> So
>> I could request my panel to execute 3 times per sec without chart repaint
>> (this could be executed by any threads) and refresh visible chart in
>> every
>> 3rd sec (this requires rendering, so calculation is done by a background
>> thread, and painting is done by main thread))?
>>
>> Any thoughts?
>>
>> Regards,
>>
>> Y
>>
>> (I know it is not doable in a day work, but I guess all short
>> term/daytraders are having trouble bacause of refresh limitations.)
>>
>>
>> --------------------------------------------------
>> From: "Julian" <juliangoodsell@xxxxxxxxxxxx>
>> Sent: Sunday, July 12, 2009 8:14 PM
>> To: <amibroker@xxxxxxxxxxxxxxx>
>> Subject: [amibroker] Per Chart Refresh Rates
>>
>>> Hi Tomasz, I thought I'd start a new thread on this topic as I think
>>> it's
>>> an interesting one.
>>>
>>> Tomasz wrote in the "Data And PlugIn Speed" thread
>>>> I may add that the concept of independent rendering from multiple
>>>> threads
>>>> although attractive from first look, it inevitably hits the wall of GDI
>>>> which in all current versions of Windows has
>>> single system-wide exclusive global lock, which means that only ONE
>>> thread
>>> can actually render at one time. This means that adding threads does not
>>> give you better performance.
>>>
>>> In response to that Tomasz,
>>> you're referring to performance on the basis of multiple charts being
>>> rendered per refresh update.
>>> E.g. If you have ten charts on screen, and they take a total of 2
>>> seconds
>>> to render, then there's little performance gain to be had by threading
>>> them because of the GDI lock. That is fine and I get that.
>>>
>>> But what I'm referring to is the ability to control which charts render
>>> when, so that all ten don't have to be updated every refresh.
>>> The problem is that in real time mode with the refresh interval set to 0
>>> in the preferences, if I have a tick chart that only takes 10ms to
>>> update,
>>> it's still only going to be rendered every 2 seconds because that's how
>>> long the other nine charts take to render, even though I don't need them
>>> to be updated multiple times per second, but maybe only once every 5
>>> seconds or even every minute.
>>>
>>> If we could set refresh rates per chart, then you could have time
>>> critical
>>> tick charts update as fast as possible, and longer timeframe or more
>>> time
>>> expensive/less critical charts only update every 5 seconds or even
>>> longer.
>>>
>>> By staggering chart updates, traders would have much greater control
>>> over
>>> performance and not waste so much processing power updating charts that
>>> don't need to be. This would trounce any other kind of performance
>>> improvement that could be gained by optimizing the rendering engine
>>> itself, and would require no threading or any real change to the current
>>> architecture that I can see.
>>>
>>> Moreover, it appears this functionality is already in AB, but just that
>>> there's no way for the user to control it.
>>> The requestTimedRefresh() function enables you to update only the chart
>>> it
>>> is applied to, so they can obviously be rendered independently of each
>>> other.
>>>
>>> The problem though is that it is not enforceable. If I specify a refresh
>>> interval of 1 in the preferences, and then requestTimedRefresh(10) on a
>>> chart, that chart still gets updated every second along with all the
>>> others, and then once more after ten seconds.
>>>
>>> Giving the option to make requestTimedRefresh() enforceable would be one
>>> way of enabling this functionality. Perhaps add an enforceable parameter
>>> to the function like:
>>> requestTimedRefresh(10, onlyvisible=True, enforceable=false).
>>> Then if I specify requestTimedRefresh(10, true, true), that chart should
>>> only update every 10 seconds, irrespective of what I've set in the
>>> preferences.
>>>
>>> Would this be as easy to implement as I think it is? If so, I think the
>>> benefits would be rather large.
>>>
>>> Jules.
>>>
>>>
>>>
>>> ------------------------------------
>>>
>>> **** IMPORTANT PLEASE READ ****
>>> This group is for the discussion between users only.
>>> This is *NOT* technical support channel.
>>>
>>> TO GET TECHNICAL SUPPORT send an e-mail directly to
>>> SUPPORT {at} amibroker.com
>>>
>>> TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
>>> http://www.amibroker.com/feedback/
>>> (submissions sent via other channels won't be considered)
>>>
>>> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
>>> http://www.amibroker.com/devlog/
>>>
>>> Yahoo! Groups Links
>>>
>>>
>>>
>>>
>>
>>
>> ------------------------------------
>>
>> **** IMPORTANT PLEASE READ ****
>> This group is for the discussion between users only.
>> This is *NOT* technical support channel.
>>
>> TO GET TECHNICAL SUPPORT send an e-mail directly to
>> SUPPORT {at} amibroker.com
>>
>> TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
>> http://www.amibroker.com/feedback/
>> (submissions sent via other channels won't be considered)
>>
>> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
>> http://www.amibroker.com/devlog/
>>
>> Yahoo! Groups Links
>>
>>
>>
>
>
> ------------------------------------
>
> **** IMPORTANT PLEASE READ ****
> This group is for the discussion between users only.
> This is *NOT* technical support channel.
>
> TO GET TECHNICAL SUPPORT send an e-mail directly to
> SUPPORT {at} amibroker.com
>
> TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
> http://www.amibroker.com/feedback/
> (submissions sent via other channels won't be considered)
>
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> Yahoo! Groups Links
>
>
>
>
------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|