|
PureBytes Links
Trading Reference Links
|
Hello,
As for DayOfWeek code - it compiled after unwrapping. Sorry for false alert.
Best regards,
Tomasz Janeczko
amibroker.com
----- Original Message -----
From: "Tomasz Janeczko" <groups@xxxxxxxxxxxxx>
To: <amibroker@xxxxxxxxxxxxxxx>
Sent: Wednesday, July 16, 2008 8:47 AM
Subject: Re: [amibroker] Re: Adventures in improving Ami Backtest speed
> Hello,
>
> Thank you very much for posting.
>
> Note that mathematically your RoundToPenny does different thing
> than Prec. Your function performs rounding (adds 0.5) while prec does
> truncation. Also when applied to negative numbers RoundToPenny may lead to incorrect results,
> for -12 input it will give -11.99 result. It is non-issue if you apply to positive-only price data.
>
> As for your Dayofweek code, it seems like some part got damaged as these two lines
> do not compile and some variables are missing:
>
> int ret = year+year/4-year/100+year/400+"-
> bed=pen+mad."[month]+day;
>
> Best regards,
> Tomasz Janeczko
> amibroker.com
> ----- Original Message -----
> From: "dloyer123" <dloyer123@xxxxxxxxx>
> To: <amibroker@xxxxxxxxxxxxxxx>
> Sent: Wednesday, July 16, 2008 4:49 AM
> Subject: [amibroker] Re: Adventures in improving Ami Backtest speed
>
>
>> Here the the dll code for fast DayOfWeek(), RoundPenny() and
>> IsOptionsWeek().
>>
>> I release this code to the public domain. Anyone may do anything
>> they want with it....
>>
>> There is a dll with this code in the file area. Look for my name.
>>
>>
>> // round to a penny
>> AmiVar RoundPenny(int NumArgs, AmiVar *ArgsTable) {
>> AmiVar result = gSite.AllocArrayResult();
>> int nSize = gSite.GetArraySize();
>> float *SrcArray = ArgsTable[0].array;
>> int j = SkipEmptyValues(nSize,SrcArray,result.array);
>>
>> // round each to a pennny
>> for (int i = j; i < nSize; i++) {
>> float val = SrcArray[i]*100+0.5f;
>> int ival = (int) val;
>> val = ival/100.0f;
>> result.array[i] = val;
>> }
>> return result;
>> }
>>
>> // day of week logic, runs over array
>> void dow(float *in, float *out, int cnt) {
>> int j = SkipEmptyValues(cnt,in, out);
>> long lastday = 0;
>> long lastret = 0;
>> for (int i = j; i < cnt; i++) {
>> long val = (long) in[i];
>> if (val == lastday) out[i] = (float) lastret;
>> else if (val == lastday+1) {
>> lastret++; lastret %= 7;
>> lastday = val;
>> out[i] = (float) lastret;
>> } else {
>> // we have to calc it the hard way
>> lastday = val;
>> int day = val % 100;
>> val /= 100;
>> int month = val % 100;
>> val /= 100;
>> int year = val + 1900;
>> if (month < 3) year--;
>> int ret = year+year/4-year/100+year/400+"-
>> bed=pen+mad."[month]+day;
>> // ret++; // now convert to sun = 0;
>> ret %= 7;
>> lastret = ret;
>> out[i] = (float) ret;
>> }
>> }
>> }
>>
>>
>>
>> // fast day of week
>> // pass in datenum
>> AmiVar FDayOfWeek(int NumArgs, AmiVar *ArgsTable) {
>> AmiVar result = gSite.AllocArrayResult();
>> int nSize = gSite.GetArraySize();
>>
>> //FILE *log = fopen("log","w");
>> //fprintf(log,"fdayofweek elements:%d\n", nSize);
>> //fclose(log);
>> float *SrcArray = ArgsTable[0].array;
>> dow(SrcArray, result.array, nSize);
>> return result;
>> }
>>
>> void iow(float *dt, float *dow, float *out, int cnt) {
>> int j = SkipEmptyValues(cnt,dt,out);
>> int ldt = 0;
>> float lval = 0.0f;
>>
>> for (int i = j; i < cnt; i++) {
>> int idtnum = (int) dt[i];
>> if (idtnum == ldt) out[i] = lval;
>> else {
>> int idow = (int) dow[i];
>> int iday = idtnum % 100;
>> int df = (12-idow)%7 + iday;
>> if (df >= 15 && df <= 21) lval = 1.0f;
>> else lval = 0.0f;
>> out[i] = lval;
>> ldt = idtnum;
>> }
>> }
>> }
>>
>>
>> // calc is options week
>> // pass datenum, dow
>> //df = (12-expwd)%7 + Day();
>> //expisow = df >= 15 && df <= 21;
>> AmiVar IsOptionsWeek(int NumArgs, AmiVar *ArgsTable) {
>> AmiVar result = gSite.AllocArrayResult();
>> int nSize = gSite.GetArraySize();
>> float *dt = ArgsTable[0].array;
>> float *dow = ArgsTable[1].array;
>> iow(dt,dow,result.array,nSize);
>> return result;
>> }
>>
>> --- In amibroker@xxxxxxxxxxxxxxx, "Tomasz Janeczko" <groups@xxx>
>> wrote:
>>>
>>> Hello,
>>>
>>> Thank you for your feedback. It is quite interesting what you
>>> are saying about DayOfWeek() and Prec().
>>>
>>> As for DayOfWeek() it is true that no optimizations were
>>> done in this function and it simply calls C runtime mktime() for
>> dates >= 1970
>>> and Windows OLE date functions for earlier dates that are
>>> not particularly fast and does so for every bar, so
>>> it could be speeded up especially if operating on intraday data
>>> when day changes infrequently.
>>>
>>> I assume that as far as DayOfWeek is considered you are just calling
>>> it once and save the result in variable.
>>>
>>> As for Prec(): Prec() uses floor() C-runtime function plus one
>> multiplication
>>> and one division per bar. In general case it is the optimum choice,
>> however
>>> more efficient ways can be found in specialized cases when for
>> example
>>> you are truncating fractional part only.
>>>
>>>
>>> Best regards,
>>> Tomasz Janeczko
>>> amibroker.com
>>> ----- Original Message -----
>>> From: "dloyer123" <dloyer123@xxx>
>>> To: <amibroker@xxxxxxxxxxxxxxx>
>>> Sent: Tuesday, July 15, 2008 6:59 PM
>>> Subject: [amibroker] Adventures in improving Ami Backtest speed
>>>
>>>
>>> > Greetings
>>> >
>>> > I just wanted to share my results in improving AmiBroker backtest
>>> > performance.
>>> >
>>> > I use 5 min intraday data over a 1,000 symbol database and
>>> > Amibroker's new walkforward support. However each walkforward
>> step
>>> > takes a long time probably due to the large size of the data
>> set.
>>> >
>>> > I have looked into ways to improve the run time performance.
>> Here
>>> > are some of the things I tried:
>>> >
>>> > 1) Buy new computer. This had the best results. Run time per
>> pass
>>> > dropped greatly. A new core 2 processor with the highest clock
>> rate
>>> > I could find was about twice as fast as my old laptop.
>>> >
>>> > 2) Use "Check AFL" to find functions that are slow and write them
>> in
>>> > C. I found that the DayOfWeek() function is very slow in
>> comparison
>>> > to other operations, so is Prec(). I moved these to C and
>> reduced my
>>> > run time a good bit. My DayOfWeek() function exploits the fact
>> that
>>> > each bar of a day is the same day of the week and the first bar
>> of
>>> > the day is probably the day after the last. It avoids finding
>> the
>>> > actual day on each bar. This saved more time than any of the
>> other
>>> > optimizations.
>>> >
>>> > 3) Rewrite the whole system in C. Sadly this only saved a about
>> 10%,
>>> > not enough to justify the risk of bugs and the trouble of
>> maintaining
>>> > the code. I went to no great lengths to optimize the code, but
>> it
>>> > goes to show how little overhead AFL adds and how highly
>> optimized
>>> > the AFL code is already.
>>> >
>>> > 4) Write a C route to cache results that do not depend on the
>> value
>>> > being optimized. Since many of the calculations dont change with
>>> > each optimization step or only depend on a single optimization
>>> > parameter, they dont need to be recalculated with each
>> optimization
>>> > step. I had high hopes for this approach and spent some time on
>> it.
>>> > I was able to get cache hit rates up over 99% on a walkforward
>> test,
>>> > but ran into memory management problems. I could overcome the
>>> > problems, but the results where disappointing. It only saved
>> about
>>> > 20% of the time per run. I suspect that there is enough overhead
>> in
>>> > setting up the stock arrays that avoiding some calculations did
>> not
>>> > make much difference. Also, it becomes hard to track what each
>>> > calculation depends on.
>>> >
>>> > It was interesting that many of the values could be compressed
>> using
>>> > simple repeat coding. 200MB was enough the cache all of the
>> values
>>> > that did not depend on any optimization parm and are re-used many
>>> > times for each optimization pass.
>>> >
>>> > 5) Improvements to AmiBroker. The new optimizer was a big help,
>> so
>>> > was the new support for QuickALF. These required no changes to
>> the
>>> > code and had a large improvement and made walkforward testing
>> much
>>> > more practical. The new optimizer allowed me to replace my own
>>> > search code that I had hacked using AFL.
>>> >
>>> > 6) Multi core support - Sadly I have not found a practical way to
>> put
>>> > the other cores on my system to work. To work with the new
>>> > optimization framework, it looks like this is best done within
>> the
>>> > ami code itself.
>>> >
>>> > 7) Really exotic stuff - I played with the CUDA api a bit. Very
>> cool
>>> > stuff. It allows of the 64 or so cores in the graphics
>> processor.
>>> > These are each able to perform one single precision floating
>> point
>>> > operation per cycle. To work, I would have to preload the
>> database
>>> > into the graphics card memory and find a way to avoid any per
>> symbol
>>> > overhead in amibroker. This would just shove the
>> buy/sell/buyprice,
>>> > etc arrays back to ami to process the trade list. If I could
>> find a
>>> > way to avoid any overhead in setting up open/high/low/close
>> arrays in
>>> > amibroker, I would do it.
>>> >
>>> > Fresh out of other ideas.
>>> >
>>> >
>>> >
>>> > ------------------------------------
>>> >
>>> > Please note that this group is for discussion between users only.
>>> >
>>> > To get support from AmiBroker please send an e-mail directly to
>>> > SUPPORT {at} amibroker.com
>>> >
>>> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
>>> > http://www.amibroker.com/devlog/
>>> >
>>> > For other support material please check also:
>>> > http://www.amibroker.com/support.html
>>> > Yahoo! Groups Links
>>> >
>>> >
>>> >
>>>
>>
>>
>>
>> ------------------------------------
>>
>> Please note that this group is for discussion between users only.
>>
>> To get support from AmiBroker please send an e-mail directly to
>> SUPPORT {at} amibroker.com
>>
>> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
>> http://www.amibroker.com/devlog/
>>
>> For other support material please check also:
>> http://www.amibroker.com/support.html
>> Yahoo! Groups Links
>>
>>
>>
>
> ------------------------------------
>
> Please note that this group is for discussion between users only.
>
> To get support from AmiBroker please send an e-mail directly to
> SUPPORT {at} amibroker.com
>
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> For other support material please check also:
> http://www.amibroker.com/support.html
> Yahoo! Groups Links
>
>
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|