|
PureBytes Links
Trading Reference Links
|
Hello,
Thank you for your feedback. It is quite interesting what you
are saying about DayOfWeek() and Prec().
As for DayOfWeek() it is true that no optimizations were
done in this function and it simply calls C runtime mktime() for dates >= 1970
and Windows OLE date functions for earlier dates that are
not particularly fast and does so for every bar, so
it could be speeded up especially if operating on intraday data
when day changes infrequently.
I assume that as far as DayOfWeek is considered you are just calling
it once and save the result in variable.
As for Prec(): Prec() uses floor() C-runtime function plus one multiplication
and one division per bar. In general case it is the optimum choice, however
more efficient ways can be found in specialized cases when for example
you are truncating fractional part only.
Best regards,
Tomasz Janeczko
amibroker.com
----- Original Message -----
From: "dloyer123" <dloyer123@xxxxxxxxx>
To: <amibroker@xxxxxxxxxxxxxxx>
Sent: Tuesday, July 15, 2008 6:59 PM
Subject: [amibroker] Adventures in improving Ami Backtest speed
> Greetings
>
> I just wanted to share my results in improving AmiBroker backtest
> performance.
>
> I use 5 min intraday data over a 1,000 symbol database and
> Amibroker's new walkforward support. However each walkforward step
> takes a long time probably due to the large size of the data set.
>
> I have looked into ways to improve the run time performance. Here
> are some of the things I tried:
>
> 1) Buy new computer. This had the best results. Run time per pass
> dropped greatly. A new core 2 processor with the highest clock rate
> I could find was about twice as fast as my old laptop.
>
> 2) Use "Check AFL" to find functions that are slow and write them in
> C. I found that the DayOfWeek() function is very slow in comparison
> to other operations, so is Prec(). I moved these to C and reduced my
> run time a good bit. My DayOfWeek() function exploits the fact that
> each bar of a day is the same day of the week and the first bar of
> the day is probably the day after the last. It avoids finding the
> actual day on each bar. This saved more time than any of the other
> optimizations.
>
> 3) Rewrite the whole system in C. Sadly this only saved a about 10%,
> not enough to justify the risk of bugs and the trouble of maintaining
> the code. I went to no great lengths to optimize the code, but it
> goes to show how little overhead AFL adds and how highly optimized
> the AFL code is already.
>
> 4) Write a C route to cache results that do not depend on the value
> being optimized. Since many of the calculations dont change with
> each optimization step or only depend on a single optimization
> parameter, they dont need to be recalculated with each optimization
> step. I had high hopes for this approach and spent some time on it.
> I was able to get cache hit rates up over 99% on a walkforward test,
> but ran into memory management problems. I could overcome the
> problems, but the results where disappointing. It only saved about
> 20% of the time per run. I suspect that there is enough overhead in
> setting up the stock arrays that avoiding some calculations did not
> make much difference. Also, it becomes hard to track what each
> calculation depends on.
>
> It was interesting that many of the values could be compressed using
> simple repeat coding. 200MB was enough the cache all of the values
> that did not depend on any optimization parm and are re-used many
> times for each optimization pass.
>
> 5) Improvements to AmiBroker. The new optimizer was a big help, so
> was the new support for QuickALF. These required no changes to the
> code and had a large improvement and made walkforward testing much
> more practical. The new optimizer allowed me to replace my own
> search code that I had hacked using AFL.
>
> 6) Multi core support - Sadly I have not found a practical way to put
> the other cores on my system to work. To work with the new
> optimization framework, it looks like this is best done within the
> ami code itself.
>
> 7) Really exotic stuff - I played with the CUDA api a bit. Very cool
> stuff. It allows of the 64 or so cores in the graphics processor.
> These are each able to perform one single precision floating point
> operation per cycle. To work, I would have to preload the database
> into the graphics card memory and find a way to avoid any per symbol
> overhead in amibroker. This would just shove the buy/sell/buyprice,
> etc arrays back to ami to process the trade list. If I could find a
> way to avoid any overhead in setting up open/high/low/close arrays in
> amibroker, I would do it.
>
> Fresh out of other ideas.
>
>
>
> ------------------------------------
>
> Please note that this group is for discussion between users only.
>
> To get support from AmiBroker please send an e-mail directly to
> SUPPORT {at} amibroker.com
>
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> For other support material please check also:
> http://www.amibroker.com/support.html
> Yahoo! Groups Links
>
>
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|