|
PureBytes Links
Trading Reference Links
|
The DayOfWeek() was the biggest gain by far.
When I get back to my system tonight, I will post the source code.
The code avoids the call to mktime() with a code snippet I found on
google that performs the calculation directly. It only needs to
perform the actual calculation on the first bar of the week and the
first bar of the month.
That sounds about right for Prec(). You might be able to trade the
division for a multiply. Ie, multiply by 0.01 rather than divide by
100. There might be an extra interger/float conversion in there. I
was surprised by how much time the "check afl" function reports for
Prec().
In any case what I was really after was to round to the nearest
penny, so it adds a half cent before taking the floor.
I'll post source code when I get home.
--- In amibroker@xxxxxxxxxxxxxxx, "Tomasz Janeczko" <groups@xxx>
wrote:
>
> Hello,
>
> Thank you for your feedback. It is quite interesting what you
> are saying about DayOfWeek() and Prec().
>
> As for DayOfWeek() it is true that no optimizations were
> done in this function and it simply calls C runtime mktime() for
dates >= 1970
> and Windows OLE date functions for earlier dates that are
> not particularly fast and does so for every bar, so
> it could be speeded up especially if operating on intraday data
> when day changes infrequently.
>
> I assume that as far as DayOfWeek is considered you are just calling
> it once and save the result in variable.
>
> As for Prec(): Prec() uses floor() C-runtime function plus one
multiplication
> and one division per bar. In general case it is the optimum choice,
however
> more efficient ways can be found in specialized cases when for
example
> you are truncating fractional part only.
>
>
> Best regards,
> Tomasz Janeczko
> amibroker.com
> ----- Original Message -----
> From: "dloyer123" <dloyer123@xxx>
> To: <amibroker@xxxxxxxxxxxxxxx>
> Sent: Tuesday, July 15, 2008 6:59 PM
> Subject: [amibroker] Adventures in improving Ami Backtest speed
>
>
> > Greetings
> >
> > I just wanted to share my results in improving AmiBroker backtest
> > performance.
> >
> > I use 5 min intraday data over a 1,000 symbol database and
> > Amibroker's new walkforward support. However each walkforward
step
> > takes a long time probably due to the large size of the data
set.
> >
> > I have looked into ways to improve the run time performance.
Here
> > are some of the things I tried:
> >
> > 1) Buy new computer. This had the best results. Run time per
pass
> > dropped greatly. A new core 2 processor with the highest clock
rate
> > I could find was about twice as fast as my old laptop.
> >
> > 2) Use "Check AFL" to find functions that are slow and write them
in
> > C. I found that the DayOfWeek() function is very slow in
comparison
> > to other operations, so is Prec(). I moved these to C and
reduced my
> > run time a good bit. My DayOfWeek() function exploits the fact
that
> > each bar of a day is the same day of the week and the first bar
of
> > the day is probably the day after the last. It avoids finding
the
> > actual day on each bar. This saved more time than any of the
other
> > optimizations.
> >
> > 3) Rewrite the whole system in C. Sadly this only saved a about
10%,
> > not enough to justify the risk of bugs and the trouble of
maintaining
> > the code. I went to no great lengths to optimize the code, but
it
> > goes to show how little overhead AFL adds and how highly
optimized
> > the AFL code is already.
> >
> > 4) Write a C route to cache results that do not depend on the
value
> > being optimized. Since many of the calculations dont change with
> > each optimization step or only depend on a single optimization
> > parameter, they dont need to be recalculated with each
optimization
> > step. I had high hopes for this approach and spent some time on
it.
> > I was able to get cache hit rates up over 99% on a walkforward
test,
> > but ran into memory management problems. I could overcome the
> > problems, but the results where disappointing. It only saved
about
> > 20% of the time per run. I suspect that there is enough overhead
in
> > setting up the stock arrays that avoiding some calculations did
not
> > make much difference. Also, it becomes hard to track what each
> > calculation depends on.
> >
> > It was interesting that many of the values could be compressed
using
> > simple repeat coding. 200MB was enough the cache all of the
values
> > that did not depend on any optimization parm and are re-used many
> > times for each optimization pass.
> >
> > 5) Improvements to AmiBroker. The new optimizer was a big help,
so
> > was the new support for QuickALF. These required no changes to
the
> > code and had a large improvement and made walkforward testing
much
> > more practical. The new optimizer allowed me to replace my own
> > search code that I had hacked using AFL.
> >
> > 6) Multi core support - Sadly I have not found a practical way to
put
> > the other cores on my system to work. To work with the new
> > optimization framework, it looks like this is best done within
the
> > ami code itself.
> >
> > 7) Really exotic stuff - I played with the CUDA api a bit. Very
cool
> > stuff. It allows of the 64 or so cores in the graphics
processor.
> > These are each able to perform one single precision floating
point
> > operation per cycle. To work, I would have to preload the
database
> > into the graphics card memory and find a way to avoid any per
symbol
> > overhead in amibroker. This would just shove the
buy/sell/buyprice,
> > etc arrays back to ami to process the trade list. If I could
find a
> > way to avoid any overhead in setting up open/high/low/close
arrays in
> > amibroker, I would do it.
> >
> > Fresh out of other ideas.
> >
> >
> >
> > ------------------------------------
> >
> > Please note that this group is for discussion between users only.
> >
> > To get support from AmiBroker please send an e-mail directly to
> > SUPPORT {at} amibroker.com
> >
> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > http://www.amibroker.com/devlog/
> >
> > For other support material please check also:
> > http://www.amibroker.com/support.html
> > Yahoo! Groups Links
> >
> >
> >
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|