[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: Freakishly fast backtest using 64 cores



PureBytes Links

Trading Reference Links

Tomasz,
  Please provide some insight here. Is this for real? If so then can
we all expect to see a future version of Amibroker that takes
advantage of this? I really want to believe this but it just seems to
good to be true. 

  Pete  :-)

--- In amibroker@xxxxxxxxxxxxxxx, "Paul Ho" <paul.tsho@xxx> wrote:
>
> I'm very interested
> could you elaborate a bit more
> What model of Nvidia chipset are you using, and with how much memory?
> Not sure exactly what you mean when you say
> It uses AmiBroker to load the symbol data and perform calculations 
> that do not depend on the optimization parameters. Once loaded into 
> video memory, repeated passes can be made with different parameters, 
> avoiding any overhead. 
> Can you give me some examples. I presume when your dll is called. AB
passes
> one or more arrays of data belonging to 1 symbol, is that true?
> Not sure exactly what the rest mean either. How many functions are you
> running in your dll, and what does each of the do?
> Great of you to share your insight.
> Cheers
> Paul.
>  
> 
> 
>   _____  
> 
> From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx]
On Behalf
> Of dloyer123
> Sent: Tuesday, 5 August 2008 9:19 AM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: [amibroker] Freakishly fast backtest using 64 cores
> 
> 
> 
> Greetings,
> 
> I ported part of my AFL backtest code to a plugin, that takes 
> advantage of the graphics math cores on the video card that are 
> normally used for 3d graphics. 
> 
> I was able to get a several thousand fold performance improvement 
> over AFL code alone.
> 
> My goal was to reduce the 25 seconds AFL code alone uses for a single 
> portfolio level back test to less than 1 second, allowing multi day 
> optimization and walkforward runs to complete in a more reasonable 
> time, and also just to see how fast I could get it to run.
> 
> The backtest runs over 1 year of 5 minute bars for about 1000 
> symbols. 1 year of data normally takes 25 seconds for AmiBroker 
> alone, or 18 seconds for 6 months of data. A typical optimization 
> run takes hundreds of these passes per walk forward step, taking 
> hours.
> 
> Using the Nvidia CUDA API, running on my mid range video card. It 
> was much faster. Much, much, much faster. How fast?
> 
> It reduced the run time from 25s to... 4.4ms. That is more than 
> 200/s! 
> 
> I didnt believe the timing when I saw it at first. So, I put 1,000 
> runs in a loop and sure enough, it ran 1,000 iterations in about 4 
> 1/2 seconds. This far exceeded my gaol or expectations.
> 
> The resulting trade list matches that obtained by the AFL version of 
> this code. 
> 
> I estimate that it is processing 32GB of bar data/sec.
> 
> Getting this to work at peak performance was tricky. Most of what I 
> have learned about code optimization does not apply. 
> 
> It uses AmiBroker to load the symbol data and perform calculations 
> that do not depend on the optimization parameters. Once loaded into 
> video memory, repeated passes can be made with different parameters, 
> avoiding any overhead. 
> 
> For non backtest/optimization runs, the code just evaluates one 
> symbol and passes the data back to AmiBroker buy/sell/short/cover 
> arrays, making it easy to test, validate and visualize the trades. 
> There is very little performance gain in this case. 
> 
> There are problems, however. To run optimizations at peak speed, I 
> can not use AmiBroker to calculate the optimization goal function. 
> So, I am in the process of writing code to match signals and 
> calculate the portfolio fitness function. Once I do this, I will be 
> able to perform full optimizations and walk forwards at 3 orders of 
> magnitude faster than is possible with AmiBroker alone.
> 
> Also, this is not general purpose code. Changing the system code 
> means changing a dll written in C. However, there is no reason that 
> this could not be made more general. 
> 
> I have made some prototypes of "Cuda" versions of basic AFL 
> functions. The idea is to queue the function calls into a definition 
> executed by a micro kernel running on the graphics cores. The result 
> would be the ability to use the full power of the graphics cores by 
> modifying AFL code to use Cuda aware versions with no changes to C 
> code. It would be an interesting, but big project.
>



------------------------------------

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/