|
PureBytes Links
Trading Reference Links
|
Nevermind - I figured it out. I am #including a file that defines buy/sell
by date. In fact, I think I will rewrite it... 8 - )
Steve
----- Original Message -----
From: "Steve Dugas" <sjdugas@xxxxxxxxxxx>
To: <amibroker@xxxxxxxxxxxxxxx>
Sent: Tuesday, July 15, 2008 6:47 PM
Subject: Re: [amibroker] Adventures in improving Ami Backtest speed
> Hi TJ - When I read this thread I went to the Code Profiler for the first
> time and profiled one of my AFL's. It tells me that DateNum is called 200
> times when in fact the only call to DateNum is
>
> Dates = DateNum();
>
> once at the beginning of the code. It is not inside a loop or anything,
> and
> even Dates variable is only used 9 times after defining it. Just wondering
> if you might have any idea why it says 200? Thanks!
>
> Steve
>
> ----- Original Message -----
> From: "Tomasz Janeczko" <groups@xxxxxxxxxxxxx>
> To: <amibroker@xxxxxxxxxxxxxxx>
> Sent: Tuesday, July 15, 2008 4:53 PM
> Subject: Re: [amibroker] Adventures in improving Ami Backtest speed
>
>
>> Hello,
>>
>> Thank you for your feedback. It is quite interesting what you
>> are saying about DayOfWeek() and Prec().
>>
>> As for DayOfWeek() it is true that no optimizations were
>> done in this function and it simply calls C runtime mktime() for dates >=
>> 1970
>> and Windows OLE date functions for earlier dates that are
>> not particularly fast and does so for every bar, so
>> it could be speeded up especially if operating on intraday data
>> when day changes infrequently.
>>
>> I assume that as far as DayOfWeek is considered you are just calling
>> it once and save the result in variable.
>>
>> As for Prec(): Prec() uses floor() C-runtime function plus one
>> multiplication
>> and one division per bar. In general case it is the optimum choice,
>> however
>> more efficient ways can be found in specialized cases when for example
>> you are truncating fractional part only.
>>
>>
>> Best regards,
>> Tomasz Janeczko
>> amibroker.com
>> ----- Original Message -----
>> From: "dloyer123" <dloyer123@xxxxxxxxx>
>> To: <amibroker@xxxxxxxxxxxxxxx>
>> Sent: Tuesday, July 15, 2008 6:59 PM
>> Subject: [amibroker] Adventures in improving Ami Backtest speed
>>
>>
>>> Greetings
>>>
>>> I just wanted to share my results in improving AmiBroker backtest
>>> performance.
>>>
>>> I use 5 min intraday data over a 1,000 symbol database and
>>> Amibroker's new walkforward support. However each walkforward step
>>> takes a long time probably due to the large size of the data set.
>>>
>>> I have looked into ways to improve the run time performance. Here
>>> are some of the things I tried:
>>>
>>> 1) Buy new computer. This had the best results. Run time per pass
>>> dropped greatly. A new core 2 processor with the highest clock rate
>>> I could find was about twice as fast as my old laptop.
>>>
>>> 2) Use "Check AFL" to find functions that are slow and write them in
>>> C. I found that the DayOfWeek() function is very slow in comparison
>>> to other operations, so is Prec(). I moved these to C and reduced my
>>> run time a good bit. My DayOfWeek() function exploits the fact that
>>> each bar of a day is the same day of the week and the first bar of
>>> the day is probably the day after the last. It avoids finding the
>>> actual day on each bar. This saved more time than any of the other
>>> optimizations.
>>>
>>> 3) Rewrite the whole system in C. Sadly this only saved a about 10%,
>>> not enough to justify the risk of bugs and the trouble of maintaining
>>> the code. I went to no great lengths to optimize the code, but it
>>> goes to show how little overhead AFL adds and how highly optimized
>>> the AFL code is already.
>>>
>>> 4) Write a C route to cache results that do not depend on the value
>>> being optimized. Since many of the calculations dont change with
>>> each optimization step or only depend on a single optimization
>>> parameter, they dont need to be recalculated with each optimization
>>> step. I had high hopes for this approach and spent some time on it.
>>> I was able to get cache hit rates up over 99% on a walkforward test,
>>> but ran into memory management problems. I could overcome the
>>> problems, but the results where disappointing. It only saved about
>>> 20% of the time per run. I suspect that there is enough overhead in
>>> setting up the stock arrays that avoiding some calculations did not
>>> make much difference. Also, it becomes hard to track what each
>>> calculation depends on.
>>>
>>> It was interesting that many of the values could be compressed using
>>> simple repeat coding. 200MB was enough the cache all of the values
>>> that did not depend on any optimization parm and are re-used many
>>> times for each optimization pass.
>>>
>>> 5) Improvements to AmiBroker. The new optimizer was a big help, so
>>> was the new support for QuickALF. These required no changes to the
>>> code and had a large improvement and made walkforward testing much
>>> more practical. The new optimizer allowed me to replace my own
>>> search code that I had hacked using AFL.
>>>
>>> 6) Multi core support - Sadly I have not found a practical way to put
>>> the other cores on my system to work. To work with the new
>>> optimization framework, it looks like this is best done within the
>>> ami code itself.
>>>
>>> 7) Really exotic stuff - I played with the CUDA api a bit. Very cool
>>> stuff. It allows of the 64 or so cores in the graphics processor.
>>> These are each able to perform one single precision floating point
>>> operation per cycle. To work, I would have to preload the database
>>> into the graphics card memory and find a way to avoid any per symbol
>>> overhead in amibroker. This would just shove the buy/sell/buyprice,
>>> etc arrays back to ami to process the trade list. If I could find a
>>> way to avoid any overhead in setting up open/high/low/close arrays in
>>> amibroker, I would do it.
>>>
>>> Fresh out of other ideas.
>>>
>>>
>>>
>>> ------------------------------------
>>>
>>> Please note that this group is for discussion between users only.
>>>
>>> To get support from AmiBroker please send an e-mail directly to
>>> SUPPORT {at} amibroker.com
>>>
>>> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
>>> http://www.amibroker.com/devlog/
>>>
>>> For other support material please check also:
>>> http://www.amibroker.com/support.html
>>> Yahoo! Groups Links
>>>
>>>
>>>
>>
>> ------------------------------------
>>
>> Please note that this group is for discussion between users only.
>>
>> To get support from AmiBroker please send an e-mail directly to
>> SUPPORT {at} amibroker.com
>>
>> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
>> http://www.amibroker.com/devlog/
>>
>> For other support material please check also:
>> http://www.amibroker.com/support.html
>> Yahoo! Groups Links
>>
>>
>>
>>
>
>
>
> ------------------------------------
>
> Please note that this group is for discussion between users only.
>
> To get support from AmiBroker please send an e-mail directly to
> SUPPORT {at} amibroker.com
>
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> For other support material please check also:
> http://www.amibroker.com/support.html
> Yahoo! Groups Links
>
>
>
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|