Hello,
Thank you very much for posting.
Note that
mathematically your RoundToPenny does different thing
than Prec. Your
function performs rounding (adds 0.5) while prec does
truncation. Also when
applied to negative numbers RoundToPenny may lead to incorrect results,
for
-12 input it will give -11.99 result. It is non-issue if you apply to
positive-only price data.
As for your Dayofweek code, it seems like
some part got damaged as these two lines
do not compile and some variables
are missing:
int ret =
year+year/4-year/100+year/400+"-
bed=pen+mad."[month]+day;
Best
regards,
Tomasz Janeczko
amibroker.com
----- Original Message -----
From: "dloyer123" <dloyer123@xxxxxxcom>
To:
<amibroker@xxxxxxxxxps.com>
Sent:
Wednesday, July 16, 2008 4:49 AM
Subject: [amibroker] Re: Adventures in
improving Ami Backtest speed
> Here the the dll code for fast
DayOfWeek(), RoundPenny() and
> IsOptionsWeek().
>
>
I release this code to the public domain. Anyone may do anything
> they
want with it....
>
> There is a dll with this code in the file
area. Look for my name.
>
>
> // round to a penny
>
AmiVar RoundPenny(int NumArgs, AmiVar *ArgsTable) {
> AmiVar result =
gSite.AllocArrayResult();
> int nSize =
gSite.GetArraySize();
> float *SrcArray =
ArgsTable[0].array;
> int j =
SkipEmptyValues(nSize,SrcArray,result.array);
>
>
// round each to a pennny
> for (int i = j; i < nSize; i++) {
>
float val = SrcArray[i]*100+0.5f;
> int ival = (int) val;
>
val = ival/100.0f;
> result.array[i] = val;
> }
>
return result;
> }
>
> // day of week logic, runs over
array
> void dow(float *in, float *out, int cnt) {
> int j =
SkipEmptyValues(cnt,in, out);
> long lastday = 0;
> long
lastret = 0;
> for (int i = j; i < cnt; i++) {
> long val =
(long) in[i];
> if (val == lastday) out[i] = (float) lastret;
>
else if (val == lastday+1) {
> lastret++; lastret %= 7;
> lastday
= val;
> out[i] = (float) lastret;
> } else {
> // we have
to calc it the hard way
> lastday = val;
> int day = val %
100;
> val /= 100;
> int month = val % 100;
> val /=
100;
> int year = val + 1900;
> if (month < 3) year--;
>
int ret = year+year/4-year/100+year/400+"-
>
bed=pen+mad."[month]+day;
> // ret++; // now convert to sun =
0;
> ret %= 7;
> lastret = ret;
> out[i] = (float)
ret;
> }
> }
> }
>
>
>
> // fast
day of week
> // pass in datenum
> AmiVar FDayOfWeek(int NumArgs,
AmiVar *ArgsTable) {
> AmiVar result =
gSite.AllocArrayResult();
> int nSize =
gSite.GetArraySize();
>
> //FILE *log =
fopen("log","w");
> //fprintf(log,"fdayofweek
elements:%d\n", nSize);
> //fclose(log);
> float
*SrcArray = ArgsTable[0].array;
> dow(SrcArray, result.array,
nSize);
> return result;
> }
>
> void iow(float *dt,
float *dow, float *out, int cnt) {
> int j =
SkipEmptyValues(cnt,dt,out);
> int ldt = 0;
> float lval
= 0.0f;
>
> for (int i = j; i < cnt; i++) {
> int idtnum
= (int) dt[i];
> if (idtnum == ldt) out[i] = lval;
> else
{
> int idow = (int) dow[i];
> int iday = idtnum % 100;
>
int df = (12-idow)%7 + iday;
> if (df >= 15 && df <= 21)
lval = 1.0f;
> else lval = 0.0f;
> out[i] = lval;
> ldt =
idtnum;
> }
> }
> }
>
>
> // calc is
options week
> // pass datenum, dow
> //df = (12-expwd)%7 + Day();
> //expisow = df >= 15 && df <= 21;
> AmiVar
IsOptionsWeek(int NumArgs, AmiVar *ArgsTable) {
> AmiVar result =
gSite.AllocArrayResult();
> int nSize =
gSite.GetArraySize();
> float *dt =
ArgsTable[0].array;
> float *dow = ArgsTable[1].array;
>
iow(dt,dow,result.array,nSize);
> return result;
>
}
>
> --- In amibroker@xxxxxxxxxps.com,
"Tomasz Janeczko" <groups@xxx>
> wrote:
>>
>>
Hello,
>>
>> Thank you for your feedback. It is quite
interesting what you
>> are saying about DayOfWeek() and
Prec().
>>
>> As for DayOfWeek() it is true that no
optimizations were
>> done in this function and it simply calls C
runtime mktime() for
> dates >= 1970
>> and Windows OLE
date functions for earlier dates that are
>> not particularly fast
and does so for every bar, so
>> it could be speeded up especially
if operating on intraday data
>> when day changes infrequently.
>>
>> I assume that as far as DayOfWeek is considered you
are just calling
>> it once and save the result in
variable.
>>
>> As for Prec(): Prec() uses floor()
C-runtime function plus one
> multiplication
>> and one
division per bar. In general case it is the optimum choice,
>
however
>> more efficient ways can be found in specialized cases when
for
> example
>> you are truncating fractional part
only.
>>
>>
>> Best regards,
>> Tomasz
Janeczko
>> amibroker.com
>> ----- Original Message -----
>> From: "dloyer123" <dloyer123@x..>
>> To:
<amibroker@xxxxxxxxxps.com>
>>
Sent: Tuesday, July 15, 2008 6:59 PM
>> Subject: [amibroker]
Adventures in improving Ami Backtest speed
>>
>>
>> > Greetings
>> >
>> > I just wanted
to share my results in improving AmiBroker backtest
>> >
performance.
>> >
>> > I use 5 min intraday data
over a 1,000 symbol database and
>> > Amibroker's new walkforward
support. However each walkforward
> step
>> > takes a long
time probably due to the large size of the data
> set.
>>
>
>> > I have looked into ways to improve the run time
performance.
> Here
>> > are some of the things I
tried:
>> >
>> > 1) Buy new computer. This had the
best results. Run time per
> pass
>> > dropped greatly. A
new core 2 processor with the highest clock
> rate
>> > I
could find was about twice as fast as my old laptop.
>> >
>> > 2) Use "Check AFL" to find functions that are slow and write
them
> in
>> > C. I found that the DayOfWeek() function is
very slow in
> comparison
>> > to other operations, so is
Prec(). I moved these to C and
> reduced my
>> > run time
a good bit. My DayOfWeek() function exploits the fact
> that
>> > each bar of a day is the same day of the week and the first
bar
> of
>> > the day is probably the day after the last.
It avoids finding
> the
>> > actual day on each bar. This
saved more time than any of the
> other
>> >
optimizations.
>> >
>> > 3) Rewrite the whole system
in C. Sadly this only saved a about
> 10%,
>> > not enough
to justify the risk of bugs and the trouble of
> maintaining
>> > the code. I went to no great lengths to optimize the code,
but
> it
>> > goes to show how little overhead AFL adds
and how highly
> optimized
>> > the AFL code is already.
>> >
>> > 4) Write a C route to cache results that
do not depend on the
> value
>> > being optimized. Since
many of the calculations dont change with
>> > each optimization
step or only depend on a single optimization
>> > parameter, they
dont need to be recalculated with each
> optimization
>> >
step. I had high hopes for this approach and spent some time on
> it.
>> > I was able to get cache hit rates up over 99% on a
walkforward
> test,
>> > but ran into memory management
problems. I could overcome the
>> > problems, but the results
where disappointing. It only saved
> about
>> > 20% of the
time per run. I suspect that there is enough overhead
> in
>>
> setting up the stock arrays that avoiding some calculations did
>
not
>> > make much difference. Also, it becomes hard to track
what each
>> > calculation depends on.
>> >
>> > It was interesting that many of the values could be
compressed
> using
>> > simple repeat coding. 200MB was
enough the cache all of the
> values
>> > that did not
depend on any optimization parm and are re-used many
>> > times
for each optimization pass.
>> >
>> > 5)
Improvements to AmiBroker. The new optimizer was a big help,
> so
>> > was the new support for QuickALF. These required no changes
to
> the
>> > code and had a large improvement and made
walkforward testing
> much
>> > more practical. The new
optimizer allowed me to replace my own
>> > search code that I
had hacked using AFL.
>> >
>> > 6) Multi core
support - Sadly I have not found a practical way to
> put
>>
> the other cores on my system to work. To work with the new
>>
> optimization framework, it looks like this is best done within
>
the
>> > ami code itself.
>> >
>> > 7)
Really exotic stuff - I played with the CUDA api a bit. Very
> cool
>> > stuff. It allows of the 64 or so cores in the graphics
> processor.
>> > These are each able to perform one
single precision floating
> point
>> > operation per
cycle. To work, I would have to preload the
> database
>>
> into the graphics card memory and find a way to avoid any per
>
symbol
>> > overhead in amibroker. This would just shove the
> buy/sell/buyprice,
>> > etc arrays back to ami to
process the trade list. If I could
> find a
>> > way to
avoid any overhead in setting up open/high/low/close
> arrays in
>> > amibroker, I would do it.
>> >
>> >
Fresh out of other ideas.
>> >
>> >
>>
>
>> >
------------------------------------
>> >
>> > Please note that this group is for discussion between users
only.
>> >
>> > To get support from AmiBroker please
send an e-mail directly to
>> > SUPPORT {at}
amibroker.com
>> >
>> > For NEW RELEASE ANNOUNCEMENTS
and other news always check DEVLOG:
>> > http://www.amibroker.com/devlog/
>>
>
>> > For other support material please check
also:
>> > http://www.amibroker.com/support.html
>>
> Yahoo! Groups Links
>> >
>> >
>>
>
>>
>
>
>
>
------------------------------------
>
> Please
note that this group is for discussion between users only.
>
> To
get support from AmiBroker please send an e-mail directly to
> SUPPORT
{at} amibroker.com
>
> For NEW RELEASE ANNOUNCEMENTS and other
news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> For other support material please check also:
> http://www.amibroker.com/support.html
>
Yahoo! Groups Links
>
>
>