Hello,
Thank you very much for posting.
Note that
mathematically your RoundToPenny does different thing
than Prec. Your
function performs rounding (adds 0.5) while prec does
truncation. Also
when applied to negative numbers RoundToPenny may lead to incorrect
results,
for -12 input it will give -11.99 result. It is non-issue if you
apply to positive-only price data.
As for your Dayofweek code, it
seems like some part got damaged as these two lines
do not compile and
some variables are missing:
int ret =
year+year/4-year/100+year/400+"-
bed=pen+mad."[month]+day;
Best
regards,
Tomasz Janeczko
amibroker.com
----- Original Message -----
From: "dloyer123" <dloyer123@xxxxxxcom>
To:
<amibroker@xxxxxxxxxps.com>
Sent:
Wednesday, July 16, 2008 4:49 AM
Subject: [amibroker] Re: Adventures in
improving Ami Backtest speed
> Here the the dll code for fast
DayOfWeek(), RoundPenny() and
> IsOptionsWeek().
>
> I release this code to the public domain. Anyone may do anything
> they want with it....
>
> There is a dll with this
code in the file area. Look for my name.
>
>
> // round
to a penny
> AmiVar RoundPenny(int NumArgs, AmiVar *ArgsTable)
{
> AmiVar result = gSite.AllocArrayResult();
> int nSize =
gSite.GetArraySize();
> float *SrcArray =
ArgsTable[0].array;
> int j =
SkipEmptyValues(nSize,SrcArray,result.array);
>
> // round each to a pennny
> for (int i = j; i < nSize;
i++) {
> float val = SrcArray[i]*100+0.5f;
> int ival =
(int) val;
> val = ival/100.0f;
> result.array[i] =
val;
> }
> return result;
> }
>
> // day of
week logic, runs over array
> void dow(float *in, float *out, int cnt)
{
> int j = SkipEmptyValues(cnt,in, out);
> long lastday =
0;
> long lastret = 0;
> for (int i = j; i < cnt; i++)
{
> long val = (long) in[i];
> if (val == lastday) out[i] =
(float) lastret;
> else if (val == lastday+1) {
> lastret++;
lastret %= 7;
> lastday = val;
> out[i] = (float)
lastret;
> } else {
> // we have to calc it the hard way
>
lastday = val;
> int day = val % 100;
> val /= 100;
> int
month = val % 100;
> val /= 100;
> int year = val +
1900;
> if (month < 3) year--;
> int ret =
year+year/4-year/100+year/400+"-
>
bed=pen+mad."[month]+day;
> // ret++; // now convert to sun
= 0;
> ret %= 7;
> lastret = ret;
> out[i] = (float)
ret;
> }
> }
> }
>
>
>
> //
fast day of week
> // pass in datenum
> AmiVar FDayOfWeek(int
NumArgs, AmiVar *ArgsTable) {
> AmiVar result =
gSite.AllocArrayResult();
> int nSize =
gSite.GetArraySize();
>
> //FILE *log =
fopen("log","w");
> //fprintf(log,"fdayofweek
elements:%d\n", nSize);
> //fclose(log);
> float
*SrcArray = ArgsTable[0].array;
> dow(SrcArray, result.array,
nSize);
> return result;
> }
>
> void iow(float
*dt, float *dow, float *out, int cnt) {
> int j =
SkipEmptyValues(cnt,dt,out);
> int ldt = 0;
> float
lval = 0.0f;
>
> for (int i = j; i < cnt; i++) {
> int
idtnum = (int) dt[i];
> if (idtnum == ldt) out[i] = lval;
> else
{
> int idow = (int) dow[i];
> int iday = idtnum % 100;
>
int df = (12-idow)%7 + iday;
> if (df >= 15 && df <= 21)
lval = 1.0f;
> else lval = 0.0f;
> out[i] = lval;
> ldt =
idtnum;
> }
> }
> }
>
>
> // calc is
options week
> // pass datenum, dow
> //df = (12-expwd)%7 +
Day();
> //expisow = df >= 15 && df <= 21;
>
AmiVar IsOptionsWeek(int NumArgs, AmiVar *ArgsTable) {
> AmiVar
result = gSite.AllocArrayResult();
> int nSize =
gSite.GetArraySize();
> float *dt =
ArgsTable[0].array;
> float *dow =
ArgsTable[1].array;
>
iow(dt,dow,result.array,nSize);
> return result;
>
}
>
> --- In amibroker@xxxxxxxxxps.com,
"Tomasz Janeczko" <groups@xxx>
> wrote:
>>
>>
Hello,
>>
>> Thank you for your feedback. It is quite
interesting what you
>> are saying about DayOfWeek() and
Prec().
>>
>> As for DayOfWeek() it is true that no
optimizations were
>> done in this function and it simply calls C
runtime mktime() for
> dates >= 1970
>> and Windows OLE
date functions for earlier dates that are
>> not particularly fast
and does so for every bar, so
>> it could be speeded up especially
if operating on intraday data
>> when day changes infrequently.
>>
>> I assume that as far as DayOfWeek is considered
you are just calling
>> it once and save the result in
variable.
>>
>> As for Prec(): Prec() uses floor()
C-runtime function plus one
> multiplication
>> and one
division per bar. In general case it is the optimum choice,
>
however
>> more efficient ways can be found in specialized cases
when for
> example
>> you are truncating fractional part
only.
>>
>>
>> Best regards,
>> Tomasz
Janeczko
>> amibroker.com
>> ----- Original Message -----
>> From: "dloyer123" <dloyer123@x..>
>> To:
<amibroker@xxxxxxxxxps.com>
>>
Sent: Tuesday, July 15, 2008 6:59 PM
>> Subject: [amibroker]
Adventures in improving Ami Backtest speed
>>
>>
>> > Greetings
>> >
>> > I just wanted
to share my results in improving AmiBroker backtest
>> >
performance.
>> >
>> > I use 5 min intraday data
over a 1,000 symbol database and
>> > Amibroker's new
walkforward support. However each walkforward
> step
>>
> takes a long time probably due to the large size of the data
>
set.
>> >
>> > I have looked into ways to improve
the run time performance.
> Here
>> > are some of the
things I tried:
>> >
>> > 1) Buy new computer. This
had the best results. Run time per
> pass
>> > dropped
greatly. A new core 2 processor with the highest clock
> rate
>> > I could find was about twice as fast as my old
laptop.
>> >
>> > 2) Use "Check AFL" to find
functions that are slow and write them
> in
>> > C. I
found that the DayOfWeek() function is very slow in
> comparison
>> > to other operations, so is Prec(). I moved these to C and
> reduced my
>> > run time a good bit. My DayOfWeek()
function exploits the fact
> that
>> > each bar of a day
is the same day of the week and the first bar
> of
>> >
the day is probably the day after the last. It avoids finding
> the
>> > actual day on each bar. This saved more time than any of
the
> other
>> > optimizations.
>> >
>> > 3) Rewrite the whole system in C. Sadly this only saved a
about
> 10%,
>> > not enough to justify the risk of bugs
and the trouble of
> maintaining
>> > the code. I went
to no great lengths to optimize the code, but
> it
>> >
goes to show how little overhead AFL adds and how highly
> optimized
>> > the AFL code is already.
>> >
>>
> 4) Write a C route to cache results that do not depend on the
>
value
>> > being optimized. Since many of the calculations dont
change with
>> > each optimization step or only depend on a
single optimization
>> > parameter, they dont need to be
recalculated with each
> optimization
>> > step. I had
high hopes for this approach and spent some time on
> it.
>> > I was able to get cache hit rates up over 99% on a
walkforward
> test,
>> > but ran into memory management
problems. I could overcome the
>> > problems, but the results
where disappointing. It only saved
> about
>> > 20% of
the time per run. I suspect that there is enough overhead
> in
>> > setting up the stock arrays that avoiding some
calculations did
> not
>> > make much difference. Also,
it becomes hard to track what each
>> > calculation depends
on.
>> >
>> > It was interesting that many of the
values could be compressed
> using
>> > simple repeat
coding. 200MB was enough the cache all of the
> values
>>
> that did not depend on any optimization parm and are re-used many
>> > times for each optimization pass.
>> >
>> > 5) Improvements to AmiBroker. The new optimizer was a big
help,
> so
>> > was the new support for QuickALF. These
required no changes to
> the
>> > code and had a large
improvement and made walkforward testing
> much
>> >
more practical. The new optimizer allowed me to replace my own
>>
> search code that I had hacked using AFL.
>> >
>>
> 6) Multi core support - Sadly I have not found a practical way to
> put
>> > the other cores on my system to work. To work
with the new
>> > optimization framework, it looks like this is
best done within
> the
>> > ami code itself.
>> >
>> > 7) Really exotic stuff - I played with
the CUDA api a bit. Very
> cool
>> > stuff. It allows of
the 64 or so cores in the graphics
> processor.
>> >
These are each able to perform one single precision floating
> point
>> > operation per cycle. To work, I would have to preload the
> database
>> > into the graphics card memory and find a
way to avoid any per
> symbol
>> > overhead in
amibroker. This would just shove the
> buy/sell/buyprice,
>> > etc arrays back to ami to process the trade list. If I
could
> find a
>> > way to avoid any overhead in setting
up open/high/low/close
> arrays in
>> > amibroker,
I would do it.
>> >
>> > Fresh out of other ideas.
>> >
>> >
>> >
>> >
------------------------------------
>> >
>> > Please note that this group is for discussion between
users only.
>> >
>> > To get support from AmiBroker
please send an e-mail directly to
>> > SUPPORT {at}
amibroker.com
>> >
>> > For NEW RELEASE
ANNOUNCEMENTS and other news always check DEVLOG:
>> > http://www.amibroker.com/devlog/
>>
>
>> > For other support material please check
also:
>> > http://www.amibroker.com/support.html
>>
> Yahoo! Groups Links
>> >
>> >
>>
>
>>
>
>
>
>
------------------------------------
>
> Please
note that this group is for discussion between users only.
>
>
To get support from AmiBroker please send an e-mail directly to
>
SUPPORT {at} amibroker.com
>
> For NEW RELEASE ANNOUNCEMENTS
and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> For other support material please check also:
> http://www.amibroker.com/support.html
>
Yahoo! Groups Links
>
>
>