|
PureBytes Links
Trading Reference Links
|
The same hardware that lets a game render life like 3d graphics.
They perform single percision floating point math at rates that are
in the relm of the super computers of a few years ago. Lots of
floating point operations per second and lots of bandwidth to
memory. The graphics cores have advanced to the point of being able
to run a subset of "c", making it programable with freely
downloadable tools.
--- In amibroker@xxxxxxxxxxxxxxx, "cstrader" <cstrader232@xxx> wrote:
>
> Re: [amibroker] Freakishly fast backtest using 64 coresYes, I'm
puzzled too.
>
> Looping should depend upon the speed of the CPU (or so I thought).
Why would putting the data in a different memory store increase
looping speed?
>
> I noticed that AB runs its second backtest much faster than the
first -- I assumed because somehow it caches all of the files on the
first run (although these are big files, and how it does that is a
mystery to me).
>
> In any case, completely fascinating!
>
> cs
>
> ----- Original Message -----
> From: Herman
> To: dloyer123
> Cc: amibroker@xxxxxxxxxxxxxxx
> Sent: Tuesday, August 05, 2008 5:09 AM
> Subject: Re: [amibroker] Freakishly fast backtest using 64 cores
>
>
> This is amazing... But, i am sorry to say, i don't understand
what you are doing...
>
>
>
>
> can you put this in layman's terms?
>
>
>
>
> best regards,
>
> herman
>
>
>
>
>
>
>
> Monday, August 4, 2008, 7:18:45 PM, you wrote:
>
>
>
>
> > Greetings,
>
>
>
>
> > I ported part of my AFL backtest code to a plugin, that takes
>
> > advantage of the graphics math cores on the video card that are
>
> > normally used for 3d graphics.
>
>
>
>
> > I was able to get a several thousand fold performance
improvement
>
> > over AFL code alone.
>
>
>
>
> > My goal was to reduce the 25 seconds AFL code alone uses for a
single
>
> > portfolio level back test to less than 1 second, allowing multi
day
>
> > optimization and walkforward runs to complete in a more
reasonable
>
> > time, and also just to see how fast I could get it to run.
>
>
>
>
> > The backtest runs over 1 year of 5 minute bars for about 1000
>
> > symbols. 1 year of data normally takes 25 seconds for
AmiBroker
>
> > alone, or 18 seconds for 6 months of data. A typical
optimization
>
> > run takes hundreds of these passes per walk forward step,
taking
>
> > hours.
>
>
>
>
> > Using the Nvidia CUDA API, running on my mid range video card.
It
>
> > was much faster. Much, much, much faster. How fast?
>
>
>
>
> > It reduced the run time from 25s to... 4.4ms. That is more
than
>
> > 200/s!
>
>
>
>
> > I didnt believe the timing when I saw it at first. So, I put
1,000
>
> > runs in a loop and sure enough, it ran 1,000 iterations in
about 4
>
> > 1/2 seconds. This far exceeded my gaol or expectations.
>
>
>
>
> > The resulting trade list matches that obtained by the AFL
version of
>
> > this code.
>
>
>
>
> > I estimate that it is processing 32GB of bar data/sec.
>
> >
>
> > Getting this to work at peak performance was tricky. Most of
what I
>
> > have learned about code optimization does not apply.
>
>
>
>
> > It uses AmiBroker to load the symbol data and perform
calculations
>
> > that do not depend on the optimization parameters. Once loaded
into
>
> > video memory, repeated passes can be made with different
parameters,
>
> > avoiding any overhead.
>
>
>
>
> > For non backtest/optimization runs, the code just evaluates one
>
> > symbol and passes the data back to AmiBroker
buy/sell/short/cover
>
> > arrays, making it easy to test, validate and visualize the
trades.
>
> > There is very little performance gain in this case.
>
>
>
>
> > There are problems, however. To run optimizations at peak
speed, I
>
> > can not use AmiBroker to calculate the optimization goal
function.
>
> > So, I am in the process of writing code to match signals and
>
> > calculate the portfolio fitness function. Once I do this, I
will be
>
> > able to perform full optimizations and walk forwards at 3
orders of
>
> > magnitude faster than is possible with AmiBroker alone.
>
>
>
>
> > Also, this is not general purpose code. Changing the system
code
>
> > means changing a dll written in C. However, there is no reason
that
>
> > this could not be made more general.
>
>
>
>
> > I have made some prototypes of "Cuda" versions of basic AFL
>
> > functions. The idea is to queue the function calls into a
definition
>
> > executed by a micro kernel running on the graphics cores. The
result
>
> > would be the ability to use the full power of the graphics
cores by
>
> > modifying AFL code to use Cuda aware versions with no changes
to C
>
> > code. It would be an interesting, but big project.
>
>
>
>
>
>
>
>
>
>
> > ------------------------------------
>
>
>
>
> > Please note that this group is for discussion between users
only.
>
>
>
>
> > To get support from AmiBroker please send an e-mail directly to
>
> > SUPPORT {at} amibroker.com
>
>
>
>
> > For NEW RELEASE ANNOUNCEMENTS and other news always check
DEVLOG:
>
> > http://www.amibroker.com/devlog/
>
>
>
>
> > For other support material please check also:
>
> > http://www.amibroker.com/support.html
>
> > Yahoo! Groups Links
>
>
>
>
> > http://groups.yahoo.com/group/amibroker/
>
>
>
>
> > Individual Email | Traditional
>
>
>
>
> > http://groups.yahoo.com/group/amibroker/join
>
> > (Yahoo! ID required)
>
>
>
>
> > mailto:amibroker-digest@xxxxxxxxxxxxxxx
>
> > mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
>
>
>
>
> > amibroker-unsubscribe@xxxxxxxxxxxxxxx
>
>
>
>
> > http://docs.yahoo.com/info/terms/
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|