|
PureBytes Links
Trading Reference Links
|
Hi TJ - When I read this thread I went to the Code Profiler for the first
time and profiled one of my AFL's. It tells me that DateNum is called 200
times when in fact the only call to DateNum is
Dates = DateNum();
once at the beginning of the code. It is not inside a loop or anything, and
even Dates variable is only used 9 times after defining it. Just wondering
if you might have any idea why it says 200? Thanks!
Steve
----- Original Message -----
From: "Tomasz Janeczko" <groups@xxxxxxxxxxxxx>
To: <amibroker@xxxxxxxxxxxxxxx>
Sent: Tuesday, July 15, 2008 4:53 PM
Subject: Re: [amibroker] Adventures in improving Ami Backtest speed
> Hello,
>
> Thank you for your feedback. It is quite interesting what you
> are saying about DayOfWeek() and Prec().
>
> As for DayOfWeek() it is true that no optimizations were
> done in this function and it simply calls C runtime mktime() for dates >=
> 1970
> and Windows OLE date functions for earlier dates that are
> not particularly fast and does so for every bar, so
> it could be speeded up especially if operating on intraday data
> when day changes infrequently.
>
> I assume that as far as DayOfWeek is considered you are just calling
> it once and save the result in variable.
>
> As for Prec(): Prec() uses floor() C-runtime function plus one
> multiplication
> and one division per bar. In general case it is the optimum choice,
> however
> more efficient ways can be found in specialized cases when for example
> you are truncating fractional part only.
>
>
> Best regards,
> Tomasz Janeczko
> amibroker.com
> ----- Original Message -----
> From: "dloyer123" <dloyer123@xxxxxxxxx>
> To: <amibroker@xxxxxxxxxxxxxxx>
> Sent: Tuesday, July 15, 2008 6:59 PM
> Subject: [amibroker] Adventures in improving Ami Backtest speed
>
>
>> Greetings
>>
>> I just wanted to share my results in improving AmiBroker backtest
>> performance.
>>
>> I use 5 min intraday data over a 1,000 symbol database and
>> Amibroker's new walkforward support. However each walkforward step
>> takes a long time probably due to the large size of the data set.
>>
>> I have looked into ways to improve the run time performance. Here
>> are some of the things I tried:
>>
>> 1) Buy new computer. This had the best results. Run time per pass
>> dropped greatly. A new core 2 processor with the highest clock rate
>> I could find was about twice as fast as my old laptop.
>>
>> 2) Use "Check AFL" to find functions that are slow and write them in
>> C. I found that the DayOfWeek() function is very slow in comparison
>> to other operations, so is Prec(). I moved these to C and reduced my
>> run time a good bit. My DayOfWeek() function exploits the fact that
>> each bar of a day is the same day of the week and the first bar of
>> the day is probably the day after the last. It avoids finding the
>> actual day on each bar. This saved more time than any of the other
>> optimizations.
>>
>> 3) Rewrite the whole system in C. Sadly this only saved a about 10%,
>> not enough to justify the risk of bugs and the trouble of maintaining
>> the code. I went to no great lengths to optimize the code, but it
>> goes to show how little overhead AFL adds and how highly optimized
>> the AFL code is already.
>>
>> 4) Write a C route to cache results that do not depend on the value
>> being optimized. Since many of the calculations dont change with
>> each optimization step or only depend on a single optimization
>> parameter, they dont need to be recalculated with each optimization
>> step. I had high hopes for this approach and spent some time on it.
>> I was able to get cache hit rates up over 99% on a walkforward test,
>> but ran into memory management problems. I could overcome the
>> problems, but the results where disappointing. It only saved about
>> 20% of the time per run. I suspect that there is enough overhead in
>> setting up the stock arrays that avoiding some calculations did not
>> make much difference. Also, it becomes hard to track what each
>> calculation depends on.
>>
>> It was interesting that many of the values could be compressed using
>> simple repeat coding. 200MB was enough the cache all of the values
>> that did not depend on any optimization parm and are re-used many
>> times for each optimization pass.
>>
>> 5) Improvements to AmiBroker. The new optimizer was a big help, so
>> was the new support for QuickALF. These required no changes to the
>> code and had a large improvement and made walkforward testing much
>> more practical. The new optimizer allowed me to replace my own
>> search code that I had hacked using AFL.
>>
>> 6) Multi core support - Sadly I have not found a practical way to put
>> the other cores on my system to work. To work with the new
>> optimization framework, it looks like this is best done within the
>> ami code itself.
>>
>> 7) Really exotic stuff - I played with the CUDA api a bit. Very cool
>> stuff. It allows of the 64 or so cores in the graphics processor.
>> These are each able to perform one single precision floating point
>> operation per cycle. To work, I would have to preload the database
>> into the graphics card memory and find a way to avoid any per symbol
>> overhead in amibroker. This would just shove the buy/sell/buyprice,
>> etc arrays back to ami to process the trade list. If I could find a
>> way to avoid any overhead in setting up open/high/low/close arrays in
>> amibroker, I would do it.
>>
>> Fresh out of other ideas.
>>
>>
>>
>> ------------------------------------
>>
>> Please note that this group is for discussion between users only.
>>
>> To get support from AmiBroker please send an e-mail directly to
>> SUPPORT {at} amibroker.com
>>
>> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
>> http://www.amibroker.com/devlog/
>>
>> For other support material please check also:
>> http://www.amibroker.com/support.html
>> Yahoo! Groups Links
>>
>>
>>
>
> ------------------------------------
>
> Please note that this group is for discussion between users only.
>
> To get support from AmiBroker please send an e-mail directly to
> SUPPORT {at} amibroker.com
>
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> For other support material please check also:
> http://www.amibroker.com/support.html
> Yahoo! Groups Links
>
>
>
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|