[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: Freakishly fast backtest using 64 cores



PureBytes Links

Trading Reference Links

Keep in mind that this is hand optimized code.  This explores the 
edge of what is currently possible.  The logic of the trading system 
is coded directly on the graphics cores.

A Dll that allowed the trading system logic to be defined in AFL 
would be much more useful.  .  

--- In amibroker@xxxxxxxxxxxxxxx, "Pete" <dryheat3@xxx> wrote:
>
> Tomasz,
>   Please provide some insight here. Is this for real? If so then can
> we all expect to see a future version of Amibroker that takes
> advantage of this? I really want to believe this but it just seems 
to
> good to be true. 
> 
>   Pete  :-)
> 
> --- In amibroker@xxxxxxxxxxxxxxx, "Paul Ho" <paul.tsho@> wrote:
> >
> > I'm very interested
> > could you elaborate a bit more
> > What model of Nvidia chipset are you using, and with how much 
memory?
> > Not sure exactly what you mean when you say
> > It uses AmiBroker to load the symbol data and perform 
calculations 
> > that do not depend on the optimization parameters. Once loaded 
into 
> > video memory, repeated passes can be made with different 
parameters, 
> > avoiding any overhead. 
> > Can you give me some examples. I presume when your dll is called. 
AB
> passes
> > one or more arrays of data belonging to 1 symbol, is that true?
> > Not sure exactly what the rest mean either. How many functions 
are you
> > running in your dll, and what does each of the do?
> > Great of you to share your insight.
> > Cheers
> > Paul.
> >  
> > 
> > 
> >   _____  
> > 
> > From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx]
> On Behalf
> > Of dloyer123
> > Sent: Tuesday, 5 August 2008 9:19 AM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: [amibroker] Freakishly fast backtest using 64 cores
> > 
> > 
> > 
> > Greetings,
> > 
> > I ported part of my AFL backtest code to a plugin, that takes 
> > advantage of the graphics math cores on the video card that are 
> > normally used for 3d graphics. 
> > 
> > I was able to get a several thousand fold performance improvement 
> > over AFL code alone.
> > 
> > My goal was to reduce the 25 seconds AFL code alone uses for a 
single 
> > portfolio level back test to less than 1 second, allowing multi 
day 
> > optimization and walkforward runs to complete in a more 
reasonable 
> > time, and also just to see how fast I could get it to run.
> > 
> > The backtest runs over 1 year of 5 minute bars for about 1000 
> > symbols. 1 year of data normally takes 25 seconds for AmiBroker 
> > alone, or 18 seconds for 6 months of data. A typical optimization 
> > run takes hundreds of these passes per walk forward step, taking 
> > hours.
> > 
> > Using the Nvidia CUDA API, running on my mid range video card. It 
> > was much faster. Much, much, much faster. How fast?
> > 
> > It reduced the run time from 25s to... 4.4ms. That is more than 
> > 200/s! 
> > 
> > I didnt believe the timing when I saw it at first. So, I put 
1,000 
> > runs in a loop and sure enough, it ran 1,000 iterations in about 
4 
> > 1/2 seconds. This far exceeded my gaol or expectations.
> > 
> > The resulting trade list matches that obtained by the AFL version 
of 
> > this code. 
> > 
> > I estimate that it is processing 32GB of bar data/sec.
> > 
> > Getting this to work at peak performance was tricky. Most of what 
I 
> > have learned about code optimization does not apply. 
> > 
> > It uses AmiBroker to load the symbol data and perform 
calculations 
> > that do not depend on the optimization parameters. Once loaded 
into 
> > video memory, repeated passes can be made with different 
parameters, 
> > avoiding any overhead. 
> > 
> > For non backtest/optimization runs, the code just evaluates one 
> > symbol and passes the data back to AmiBroker buy/sell/short/cover 
> > arrays, making it easy to test, validate and visualize the 
trades. 
> > There is very little performance gain in this case. 
> > 
> > There are problems, however. To run optimizations at peak speed, 
I 
> > can not use AmiBroker to calculate the optimization goal 
function. 
> > So, I am in the process of writing code to match signals and 
> > calculate the portfolio fitness function. Once I do this, I will 
be 
> > able to perform full optimizations and walk forwards at 3 orders 
of 
> > magnitude faster than is possible with AmiBroker alone.
> > 
> > Also, this is not general purpose code. Changing the system code 
> > means changing a dll written in C. However, there is no reason 
that 
> > this could not be made more general. 
> > 
> > I have made some prototypes of "Cuda" versions of basic AFL 
> > functions. The idea is to queue the function calls into a 
definition 
> > executed by a micro kernel running on the graphics cores. The 
result 
> > would be the ability to use the full power of the graphics cores 
by 
> > modifying AFL code to use Cuda aware versions with no changes to 
C 
> > code. It would be an interesting, but big project.
> >
>



------------------------------------

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/