|
PureBytes Links
Trading Reference Links
|
Here the the dll code for fast DayOfWeek(), RoundPenny() and
IsOptionsWeek().
I release this code to the public domain. Anyone may do anything
they want with it....
There is a dll with this code in the file area. Look for my name.
// round to a penny
AmiVar RoundPenny(int NumArgs, AmiVar *ArgsTable) {
AmiVar result = gSite.AllocArrayResult();
int nSize = gSite.GetArraySize();
float *SrcArray = ArgsTable[0].array;
int j = SkipEmptyValues(nSize,SrcArray,result.array);
// round each to a pennny
for (int i = j; i < nSize; i++) {
float val = SrcArray[i]*100+0.5f;
int ival = (int) val;
val = ival/100.0f;
result.array[i] = val;
}
return result;
}
// day of week logic, runs over array
void dow(float *in, float *out, int cnt) {
int j = SkipEmptyValues(cnt,in, out);
long lastday = 0;
long lastret = 0;
for (int i = j; i < cnt; i++) {
long val = (long) in[i];
if (val == lastday) out[i] = (float) lastret;
else if (val == lastday+1) {
lastret++; lastret %= 7;
lastday = val;
out[i] = (float) lastret;
} else {
// we have to calc it the hard way
lastday = val;
int day = val % 100;
val /= 100;
int month = val % 100;
val /= 100;
int year = val + 1900;
if (month < 3) year--;
int ret = year+year/4-year/100+year/400+"-
bed=pen+mad."[month]+day;
// ret++; // now convert to sun = 0;
ret %= 7;
lastret = ret;
out[i] = (float) ret;
}
}
}
// fast day of week
// pass in datenum
AmiVar FDayOfWeek(int NumArgs, AmiVar *ArgsTable) {
AmiVar result = gSite.AllocArrayResult();
int nSize = gSite.GetArraySize();
//FILE *log = fopen("log","w");
//fprintf(log,"fdayofweek elements:%d\n", nSize);
//fclose(log);
float *SrcArray = ArgsTable[0].array;
dow(SrcArray, result.array, nSize);
return result;
}
void iow(float *dt, float *dow, float *out, int cnt) {
int j = SkipEmptyValues(cnt,dt,out);
int ldt = 0;
float lval = 0.0f;
for (int i = j; i < cnt; i++) {
int idtnum = (int) dt[i];
if (idtnum == ldt) out[i] = lval;
else {
int idow = (int) dow[i];
int iday = idtnum % 100;
int df = (12-idow)%7 + iday;
if (df >= 15 && df <= 21) lval = 1.0f;
else lval = 0.0f;
out[i] = lval;
ldt = idtnum;
}
}
}
// calc is options week
// pass datenum, dow
//df = (12-expwd)%7 + Day();
//expisow = df >= 15 && df <= 21;
AmiVar IsOptionsWeek(int NumArgs, AmiVar *ArgsTable) {
AmiVar result = gSite.AllocArrayResult();
int nSize = gSite.GetArraySize();
float *dt = ArgsTable[0].array;
float *dow = ArgsTable[1].array;
iow(dt,dow,result.array,nSize);
return result;
}
--- In amibroker@xxxxxxxxxxxxxxx, "Tomasz Janeczko" <groups@xxx>
wrote:
>
> Hello,
>
> Thank you for your feedback. It is quite interesting what you
> are saying about DayOfWeek() and Prec().
>
> As for DayOfWeek() it is true that no optimizations were
> done in this function and it simply calls C runtime mktime() for
dates >= 1970
> and Windows OLE date functions for earlier dates that are
> not particularly fast and does so for every bar, so
> it could be speeded up especially if operating on intraday data
> when day changes infrequently.
>
> I assume that as far as DayOfWeek is considered you are just calling
> it once and save the result in variable.
>
> As for Prec(): Prec() uses floor() C-runtime function plus one
multiplication
> and one division per bar. In general case it is the optimum choice,
however
> more efficient ways can be found in specialized cases when for
example
> you are truncating fractional part only.
>
>
> Best regards,
> Tomasz Janeczko
> amibroker.com
> ----- Original Message -----
> From: "dloyer123" <dloyer123@xxx>
> To: <amibroker@xxxxxxxxxxxxxxx>
> Sent: Tuesday, July 15, 2008 6:59 PM
> Subject: [amibroker] Adventures in improving Ami Backtest speed
>
>
> > Greetings
> >
> > I just wanted to share my results in improving AmiBroker backtest
> > performance.
> >
> > I use 5 min intraday data over a 1,000 symbol database and
> > Amibroker's new walkforward support. However each walkforward
step
> > takes a long time probably due to the large size of the data
set.
> >
> > I have looked into ways to improve the run time performance.
Here
> > are some of the things I tried:
> >
> > 1) Buy new computer. This had the best results. Run time per
pass
> > dropped greatly. A new core 2 processor with the highest clock
rate
> > I could find was about twice as fast as my old laptop.
> >
> > 2) Use "Check AFL" to find functions that are slow and write them
in
> > C. I found that the DayOfWeek() function is very slow in
comparison
> > to other operations, so is Prec(). I moved these to C and
reduced my
> > run time a good bit. My DayOfWeek() function exploits the fact
that
> > each bar of a day is the same day of the week and the first bar
of
> > the day is probably the day after the last. It avoids finding
the
> > actual day on each bar. This saved more time than any of the
other
> > optimizations.
> >
> > 3) Rewrite the whole system in C. Sadly this only saved a about
10%,
> > not enough to justify the risk of bugs and the trouble of
maintaining
> > the code. I went to no great lengths to optimize the code, but
it
> > goes to show how little overhead AFL adds and how highly
optimized
> > the AFL code is already.
> >
> > 4) Write a C route to cache results that do not depend on the
value
> > being optimized. Since many of the calculations dont change with
> > each optimization step or only depend on a single optimization
> > parameter, they dont need to be recalculated with each
optimization
> > step. I had high hopes for this approach and spent some time on
it.
> > I was able to get cache hit rates up over 99% on a walkforward
test,
> > but ran into memory management problems. I could overcome the
> > problems, but the results where disappointing. It only saved
about
> > 20% of the time per run. I suspect that there is enough overhead
in
> > setting up the stock arrays that avoiding some calculations did
not
> > make much difference. Also, it becomes hard to track what each
> > calculation depends on.
> >
> > It was interesting that many of the values could be compressed
using
> > simple repeat coding. 200MB was enough the cache all of the
values
> > that did not depend on any optimization parm and are re-used many
> > times for each optimization pass.
> >
> > 5) Improvements to AmiBroker. The new optimizer was a big help,
so
> > was the new support for QuickALF. These required no changes to
the
> > code and had a large improvement and made walkforward testing
much
> > more practical. The new optimizer allowed me to replace my own
> > search code that I had hacked using AFL.
> >
> > 6) Multi core support - Sadly I have not found a practical way to
put
> > the other cores on my system to work. To work with the new
> > optimization framework, it looks like this is best done within
the
> > ami code itself.
> >
> > 7) Really exotic stuff - I played with the CUDA api a bit. Very
cool
> > stuff. It allows of the 64 or so cores in the graphics
processor.
> > These are each able to perform one single precision floating
point
> > operation per cycle. To work, I would have to preload the
database
> > into the graphics card memory and find a way to avoid any per
symbol
> > overhead in amibroker. This would just shove the
buy/sell/buyprice,
> > etc arrays back to ami to process the trade list. If I could
find a
> > way to avoid any overhead in setting up open/high/low/close
arrays in
> > amibroker, I would do it.
> >
> > Fresh out of other ideas.
> >
> >
> >
> > ------------------------------------
> >
> > Please note that this group is for discussion between users only.
> >
> > To get support from AmiBroker please send an e-mail directly to
> > SUPPORT {at} amibroker.com
> >
> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > http://www.amibroker.com/devlog/
> >
> > For other support material please check also:
> > http://www.amibroker.com/support.html
> > Yahoo! Groups Links
> >
> >
> >
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|