|
PureBytes Links
Trading Reference Links
|
Amazing... thanks so much for bringing this to our attention.
----- Original Message -----
From: "dloyer123" <dloyer123@xxxxxxxxx>
To: <amibroker@xxxxxxxxxxxxxxx>
Sent: Tuesday, August 05, 2008 10:06 AM
Subject: [amibroker] Re: Freakishly fast backtest using 64 cores
> The same hardware that lets a game render life like 3d graphics.
> They perform single percision floating point math at rates that are
> in the relm of the super computers of a few years ago. Lots of
> floating point operations per second and lots of bandwidth to
> memory. The graphics cores have advanced to the point of being able
> to run a subset of "c", making it programable with freely
> downloadable tools.
>
>
>
>
>
> --- In amibroker@xxxxxxxxxxxxxxx, "cstrader" <cstrader232@xxx> wrote:
>>
>> Re: [amibroker] Freakishly fast backtest using 64 coresYes, I'm
> puzzled too.
>>
>> Looping should depend upon the speed of the CPU (or so I thought).
> Why would putting the data in a different memory store increase
> looping speed?
>>
>> I noticed that AB runs its second backtest much faster than the
> first -- I assumed because somehow it caches all of the files on the
> first run (although these are big files, and how it does that is a
> mystery to me).
>>
>> In any case, completely fascinating!
>>
>> cs
>>
>> ----- Original Message -----
>> From: Herman
>> To: dloyer123
>> Cc: amibroker@xxxxxxxxxxxxxxx
>> Sent: Tuesday, August 05, 2008 5:09 AM
>> Subject: Re: [amibroker] Freakishly fast backtest using 64 cores
>>
>>
>> This is amazing... But, i am sorry to say, i don't understand
> what you are doing...
>>
>>
>>
>>
>> can you put this in layman's terms?
>>
>>
>>
>>
>> best regards,
>>
>> herman
>>
>>
>>
>>
>>
>>
>>
>> Monday, August 4, 2008, 7:18:45 PM, you wrote:
>>
>>
>>
>>
>> > Greetings,
>>
>>
>>
>>
>> > I ported part of my AFL backtest code to a plugin, that takes
>>
>> > advantage of the graphics math cores on the video card that are
>>
>> > normally used for 3d graphics.
>>
>>
>>
>>
>> > I was able to get a several thousand fold performance
> improvement
>>
>> > over AFL code alone.
>>
>>
>>
>>
>> > My goal was to reduce the 25 seconds AFL code alone uses for a
> single
>>
>> > portfolio level back test to less than 1 second, allowing multi
> day
>>
>> > optimization and walkforward runs to complete in a more
> reasonable
>>
>> > time, and also just to see how fast I could get it to run.
>>
>>
>>
>>
>> > The backtest runs over 1 year of 5 minute bars for about 1000
>>
>> > symbols. 1 year of data normally takes 25 seconds for
> AmiBroker
>>
>> > alone, or 18 seconds for 6 months of data. A typical
> optimization
>>
>> > run takes hundreds of these passes per walk forward step,
> taking
>>
>> > hours.
>>
>>
>>
>>
>> > Using the Nvidia CUDA API, running on my mid range video card.
> It
>>
>> > was much faster. Much, much, much faster. How fast?
>>
>>
>>
>>
>> > It reduced the run time from 25s to... 4.4ms. That is more
> than
>>
>> > 200/s!
>>
>>
>>
>>
>> > I didnt believe the timing when I saw it at first. So, I put
> 1,000
>>
>> > runs in a loop and sure enough, it ran 1,000 iterations in
> about 4
>>
>> > 1/2 seconds. This far exceeded my gaol or expectations.
>>
>>
>>
>>
>> > The resulting trade list matches that obtained by the AFL
> version of
>>
>> > this code.
>>
>>
>>
>>
>> > I estimate that it is processing 32GB of bar data/sec.
>>
>> >
>>
>> > Getting this to work at peak performance was tricky. Most of
> what I
>>
>> > have learned about code optimization does not apply.
>>
>>
>>
>>
>> > It uses AmiBroker to load the symbol data and perform
> calculations
>>
>> > that do not depend on the optimization parameters. Once loaded
> into
>>
>> > video memory, repeated passes can be made with different
> parameters,
>>
>> > avoiding any overhead.
>>
>>
>>
>>
>> > For non backtest/optimization runs, the code just evaluates one
>>
>> > symbol and passes the data back to AmiBroker
> buy/sell/short/cover
>>
>> > arrays, making it easy to test, validate and visualize the
> trades.
>>
>> > There is very little performance gain in this case.
>>
>>
>>
>>
>> > There are problems, however. To run optimizations at peak
> speed, I
>>
>> > can not use AmiBroker to calculate the optimization goal
> function.
>>
>> > So, I am in the process of writing code to match signals and
>>
>> > calculate the portfolio fitness function. Once I do this, I
> will be
>>
>> > able to perform full optimizations and walk forwards at 3
> orders of
>>
>> > magnitude faster than is possible with AmiBroker alone.
>>
>>
>>
>>
>> > Also, this is not general purpose code. Changing the system
> code
>>
>> > means changing a dll written in C. However, there is no reason
> that
>>
>> > this could not be made more general.
>>
>>
>>
>>
>> > I have made some prototypes of "Cuda" versions of basic AFL
>>
>> > functions. The idea is to queue the function calls into a
> definition
>>
>> > executed by a micro kernel running on the graphics cores. The
> result
>>
>> > would be the ability to use the full power of the graphics
> cores by
>>
>> > modifying AFL code to use Cuda aware versions with no changes
> to C
>>
>> > code. It would be an interesting, but big project.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> > ------------------------------------
>>
>>
>>
>>
>> > Please note that this group is for discussion between users
> only.
>>
>>
>>
>>
>> > To get support from AmiBroker please send an e-mail directly to
>>
>> > SUPPORT {at} amibroker.com
>>
>>
>>
>>
>> > For NEW RELEASE ANNOUNCEMENTS and other news always check
> DEVLOG:
>>
>> > http://www.amibroker.com/devlog/
>>
>>
>>
>>
>> > For other support material please check also:
>>
>> > http://www.amibroker.com/support.html
>>
>> > Yahoo! Groups Links
>>
>>
>>
>>
>> > http://groups.yahoo.com/group/amibroker/
>>
>>
>>
>>
>> > Individual Email | Traditional
>>
>>
>>
>>
>> > http://groups.yahoo.com/group/amibroker/join
>>
>> > (Yahoo! ID required)
>>
>>
>>
>>
>> > mailto:amibroker-digest@xxxxxxxxxxxxxxx
>>
>> > mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
>>
>>
>>
>>
>> > amibroker-unsubscribe@xxxxxxxxxxxxxxx
>>
>>
>>
>>
>> > http://docs.yahoo.com/info/terms/
>>
>
>
>
> ------------------------------------
>
> Please note that this group is for discussion between users only.
>
> To get support from AmiBroker please send an e-mail directly to
> SUPPORT {at} amibroker.com
>
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> For other support material please check also:
> http://www.amibroker.com/support.html
> Yahoo! Groups Links
>
>
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|