|
PureBytes Links
Trading Reference Links
|
Greetings
I just wanted to share my results in improving AmiBroker backtest
performance.
I use 5 min intraday data over a 1,000 symbol database and
Amibroker's new walkforward support. However each walkforward step
takes a long time probably due to the large size of the data set.
I have looked into ways to improve the run time performance. Here
are some of the things I tried:
1) Buy new computer. This had the best results. Run time per pass
dropped greatly. A new core 2 processor with the highest clock rate
I could find was about twice as fast as my old laptop.
2) Use "Check AFL" to find functions that are slow and write them in
C. I found that the DayOfWeek() function is very slow in comparison
to other operations, so is Prec(). I moved these to C and reduced my
run time a good bit. My DayOfWeek() function exploits the fact that
each bar of a day is the same day of the week and the first bar of
the day is probably the day after the last. It avoids finding the
actual day on each bar. This saved more time than any of the other
optimizations.
3) Rewrite the whole system in C. Sadly this only saved a about 10%,
not enough to justify the risk of bugs and the trouble of maintaining
the code. I went to no great lengths to optimize the code, but it
goes to show how little overhead AFL adds and how highly optimized
the AFL code is already.
4) Write a C route to cache results that do not depend on the value
being optimized. Since many of the calculations dont change with
each optimization step or only depend on a single optimization
parameter, they dont need to be recalculated with each optimization
step. I had high hopes for this approach and spent some time on it.
I was able to get cache hit rates up over 99% on a walkforward test,
but ran into memory management problems. I could overcome the
problems, but the results where disappointing. It only saved about
20% of the time per run. I suspect that there is enough overhead in
setting up the stock arrays that avoiding some calculations did not
make much difference. Also, it becomes hard to track what each
calculation depends on.
It was interesting that many of the values could be compressed using
simple repeat coding. 200MB was enough the cache all of the values
that did not depend on any optimization parm and are re-used many
times for each optimization pass.
5) Improvements to AmiBroker. The new optimizer was a big help, so
was the new support for QuickALF. These required no changes to the
code and had a large improvement and made walkforward testing much
more practical. The new optimizer allowed me to replace my own
search code that I had hacked using AFL.
6) Multi core support - Sadly I have not found a practical way to put
the other cores on my system to work. To work with the new
optimization framework, it looks like this is best done within the
ami code itself.
7) Really exotic stuff - I played with the CUDA api a bit. Very cool
stuff. It allows of the 64 or so cores in the graphics processor.
These are each able to perform one single precision floating point
operation per cycle. To work, I would have to preload the database
into the graphics card memory and find a way to avoid any per symbol
overhead in amibroker. This would just shove the buy/sell/buyprice,
etc arrays back to ami to process the trade list. If I could find a
way to avoid any overhead in setting up open/high/low/close arrays in
amibroker, I would do it.
Fresh out of other ideas.
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|