[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [amibroker] Freakishly fast backtest using 64 cores



PureBytes Links

Trading Reference Links

Congratulations!

My jaw is planted firmly on the floor between my feet --and I am  
standing up!

I just love it when somebody rips the cardboard box completely apart  
and there is nothing left but confetti!

Thinking cap time for a lot of people.

Hints of things to come.

Thanks,

Dennis


On Aug 4, 2008, at 7:18 PM, dloyer123 wrote:

> Greetings,
>
> I ported part of my AFL backtest code to a plugin, that takes
> advantage of the graphics math cores on the video card that are
> normally used for 3d graphics.
>
> I was able to get a several thousand fold performance improvement
> over AFL code alone.
>
> My goal was to reduce the 25 seconds AFL code alone uses for a single
> portfolio level back test to less than 1 second, allowing multi day
> optimization and walkforward runs to complete in a more reasonable
> time, and also just to see how fast I could get it to run.
>
> The backtest runs over 1 year of 5 minute bars for about 1000
> symbols.  1 year of data normally takes 25 seconds for AmiBroker
> alone, or 18 seconds for 6 months of data.  A typical optimization
> run takes hundreds of these passes per walk forward step, taking
> hours.
>
> Using the Nvidia CUDA API, running on my mid range video card.  It
> was much faster.  Much, much, much faster.  How fast?
>
> It reduced the run time from 25s to... 4.4ms.  That is more than
> 200/s!
>
> I didnt believe the timing when I saw it at first.  So, I put 1,000
> runs in a loop and sure enough, it ran 1,000 iterations in about 4
> 1/2 seconds.  This far exceeded my gaol or expectations.
>
> The resulting trade list matches that obtained by the AFL version of
> this code.
>
> I estimate that it is processing 32GB of bar data/sec.
>
> Getting this to work at peak performance was tricky.  Most of what I
> have learned about code optimization does not apply.
>
> It uses AmiBroker to load the symbol data and perform calculations
> that do not depend on the optimization parameters.  Once loaded into
> video memory, repeated passes can be made with different parameters,
> avoiding any overhead.
>
> For non backtest/optimization runs, the code just evaluates one
> symbol and passes the data back to AmiBroker buy/sell/short/cover
> arrays, making it easy to test, validate and visualize the trades.
> There is very little performance gain in this case.
>
> There are problems, however.  To run optimizations at peak speed, I
> can not use AmiBroker to calculate the optimization goal function.
> So, I am in the process of writing code to match signals and
> calculate the portfolio fitness function.  Once I do this, I will be
> able to perform full optimizations and walk forwards at 3 orders of
> magnitude faster than is possible with AmiBroker alone.
>
> Also, this is not general purpose code.  Changing the system code
> means changing a dll written in C.  However, there is no reason that
> this could not be made more general.
>
> I have made some prototypes of "Cuda" versions of basic AFL
> functions.  The idea is to queue the function calls into a definition
> executed by a micro kernel running on the graphics cores.  The result
> would be the ability to use the full power of the graphics cores by
> modifying AFL code to use Cuda aware versions with no changes to C
> code. It would be an interesting, but big project.


------------------------------------

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/