[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: More on Database Structure and Local Storage



PureBytes Links

Trading Reference Links

Ken,
There is no need to have target.isDirty in the loop. once per symbol 
is quite enough.
Progster, you're correct in your interpretation. The only correction 
needed is that Tomasz first posted the original code, I added the 
storage part.

--- In amibroker@xxxxxxxxxxxxxxx, Ken Close <ken45140@xxx> wrote:
>
> Tomasz:
> 
> This is a great suggestion.  I will work on this further.
> 
> I am so very impressed with the speed of the rawbacktester.
> 
> After getting my FT db to local storage finally, I was able to 
repeat two
> tests.
> 
> Writing OI field using N^2 logic code snippit by Paul Ho.
> Time to complete 426 symbols using combination of two indicators = 
7min
> 25sec.
> All sums of ordinal values written successfully to OI.
> 
> Test using custom backtester, writing directly to OI field, code 
again
> courtesy of Paul Ho (listed below for interest)
> Time to complete same 426 symbols using combination of two 
indicators = 1min
> 42sec.
> All sums of ordinal values written successfully to OI.
> 
> Interestingly, after retrieving the OI values and writing them to 
csv file,
> combining both results into one csv file (Test1 and Test2), two 
things were
> obvious: the values written to the OI field were absolutely 
different
> integers from both tests, and, the sort order of the OI field 
contents put
> both lists in exactly the same order!
> 
> 
> Ticker	 Date/Time	 Price          	 OI	 
	Ticker
> Date/Time	 Price	 OI	
> AMGN 	7/3/2008 	50.84	 2	 	AMGN 	7/3/2008
> 50.84	 4	
> SDS 	7/3/2008 	68.61	 8	 	SDS 	7/3/2008
> 68.61	 10	
> JBL 	7/3/2008 	16.29	 16	 	JBL 	7/3/2008
> 16.29	 16	
> NBR 	7/3/2008 	48.71	 20	 	NBR 	7/3/2008
> 48.71	 18	
> KG	 7/3/2008 	10.58	 30	 	KG	 7/3/2008
> 10.58	 17	
> BAX 	7/3/2008 	64.12	 34	 	BAX 	7/3/2008
> 64.12	 43	
> EDS 	7/3/2008 	24.7	 36	 	EDS 	7/3/2008 
	24.7
> 52	
> GENZ 	7/3/2008 	73.03	 37	 	GENZ 	7/3/2008
> 73.03	 39	
> 
> With your suggestion of the artificial ticker symbol and the 
apparent
> lightning speed of the custombacktester (used for ranking and 
writing), my
> hopes are alive again that I can do my application within FT, 
without
> writing to external files, and at speeds that may allow me a 
reasonable time
> to complete the full database.
> 
> Wow, the happiness of great results coming together.
> 
> Many thanks for your help.
> 
> Ken
> 
> 
> 
> For those interested in the code written for this test by Paul Ho, 
here it
> is.  Tomasz has said several times that the ranking done in the
> custombacktester is fast and he certainly has understated the case 
here,
> 
> run = Optimize("run", 1, 1, 2, 1);
> 
> function ranking(Ordinal)
> 
> {
> 
>     switch(Ordinal)
> 
>     {
> 
>     case 1:
> 
>         res = ROC(C, 14) + 1000;
> 
>         break;
> 
>     case 2:
> 
>         res = RSI(14);
> 
>         break;
> 
>     default:
> 
>         res = EMA(C, 50);
> 
>         break;
> 
>     }
> 
>     return res;
> 
> }
> 
> PositionScore = ranking(run); // WHAT YOU WANT TO RANK 
> 
>  
> 
> SetOption("MaxOpenPositions", 50 ); //AB only keeps 2* maxpos top 
rank
> signals
> 
> SetBacktestMode( backtestRegularRaw ); 
> 
> Buy=1; 
> 
> Sell= Short = Cover = 0; 
> 
> ab = CreateObject("broker.Application");
> 
> target = ab.Stocks(Name());
> 
> target.isDirty = True;
> 
> SetCustomBacktestProc(""); 
> 
> if( Status("action")==actionPortfolio ) 
> 
> { 
> 
>     bo = GetBacktesterObject(); 
> 
>     bo.PreProcess(); 
> 
>     dt = DateTime(); 
> 
> //  fh = fopen("output.csv", "w"); uncomment these if you want to 
save it to
> a csv file as well
> 
> //  fputs("Symbol,Date,Rank\n", fh);
> 
>     for( i = 0; i < BarCount; i++ ) 
> 
>     { 
> 
>         k = 1;   
> 
>         for( sig = bo.GetFirstSignal( i ); sig; sig = 
bo.GetNextSignal( i )
> ) 
> 
>         { 
> 
> //         Line = sig.Symbol + "," + DateTimeToStr(dt[i]) + "," + k 
+ "\n";
> 
> //         fputs(Line, fh);
> 
>            target = ab.Stocks(sig.Symbol);
> 
>            qt = target.Quotations(DateTimeToStr(dt[i]));
> 
>            if(qt)
> 
>            {
> 
>                if(run == 1)
> 
>                    qt.OpenInt = k;
> 
>                else
> 
>                    qt.OpenInt += k;
> 
>            }
> 
>            target.isDirty = True;
> 
>        k++; 
> 
>         } 
> 
>     } 
> 
>  
> 
>     bo.PostProcess(); 
> 
> //  if(fh)fclose(fh);
> 
>  
> 
> 
> 
> -----Original Message-----
> From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx] 
On Behalf
> Of Tomasz Janeczko
> Sent: Tuesday, July 08, 2008 1:16 PM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: Re: [amibroker] Re: More on Database Structure and Local 
Storage
> 
> Ken,
> 
> Instead of writing to the very same symbol OI field, you may 
consider
> writing to separate 'artificial' symbol (like composite).
> That way you will have both: direct access to FT and ability to 
store what
> you need.
> 
> The only difference in the code is that instead of using actual 
symbol, you
> use artificial one (say with tilde in front).
> 
> Instead of this:
>       for( sig = bo.GetFirstSignal( i ); sig; sig = bo.GetNextSignal
( i ) )
>       {
>          target = ab.Stocks(sig.Symbol);
>          qt = target.Quotations(i);
>          if(qt)
>          {
> 
> 
> You would need this:
> 
>       for( sig = bo.GetFirstSignal( i ); sig; sig = bo.GetNextSignal
( i ) )
>       {
>          target = ab.Stocks.Add("~" + sig.Symbol); // create 
artificial
> symbol
>          target.DataSource = 1; // use only local DB
>          qts = target.Quotations;
>          qt = qts.Add(DateTimeToStr(dt[i]));
>          if(qt)
>          {
> 
> 
> Best regards,
> Tomasz Janeczko
> amibroker.com
> ----- Original Message -----
> From: "Ken Close" <ken45140@xxx>
> To: <amibroker@xxxxxxxxxxxxxxx>
> Sent: Tuesday, July 08, 2008 6:59 PM
> Subject: RE: [amibroker] Re: More on Database Structure and Local 
Storage
> 
> 
> > Barry:
> >
> > FT has its own datafiles and AB reads these files via the 
FastTrack
> PlugIn.
> > I want to write a ranking score into the OI field of each symbol, 
as the
> > calculation of the ranking symbol takes a long time to do itself.
> >
> > As I have learned, you copy to the local database by doing a Scan 
over the
> > entire external database with local set as the source; then symbol
> contents
> > are copied.  That is all fine, and I could do my calc and stick 
it into OI
> > field, but then the next time I need to update the local 
database, the OI
> > fields will be overwritten.
> >
> > Sort of circular.
> >
> > Ken
> >
> > -----Original Message-----
> > From: amibroker@xxxxxxxxxxxxxxx 
[mailto:amibroker@xxxxxxxxxxxxxxx] On
> Behalf
> > Of Barry Scarborough
> > Sent: Tuesday, July 08, 2008 8:12 AM
> > To: amibroker@xxxxxxxxxxxxxxx
> > Subject: [amibroker] Re: More on Database Structure and Local 
Storage
> >
> > There is a parameter for each symbol, Symbol > Information > Use 
only
> local
> > database, yes or no. But that will only keep the data from being 
back
> filled
> > if set to yes and you may not want to do that. If I read your 
note right
> you
> > want to import the data into AB and then keep it updated from 
some data
> > source, maybe FT. If so the parameter has to be set to no.
> >
> > How did you copy or import the FT data into AB? AB is kind of 
fussy about
> > what you put in their database. If you import the data 
incorrectly it will
> > blow up every time you try to use the symbol.
> >
> > What do you want to write into the OI field? When are you trying 
to write
> > it?
> >
> > Barry
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "Ken Close" <ken45140@> wrote:
> >>
> >> Can someone explain to me how I make the FastTrack database a 
local
> > one? 
> >> 
> >> Background: I have had a FT db for years, with local storage
> > disabled?  I
> >> want to write to the OI field and preserve the written data. I
> > checked
> >> "Enable Local Storage" but nothing different happened.
> >> 
> >> Next, I tried to scan all symbols with a simple formula (I
> > remembered having
> >> to do that years ago with a TC2000 database that came on a CD--I
> > thought the
> >> same kind of step was needed.).   Amibroker got hung on a single
> > symbol and
> >> I had to End Program.  AB would lock if I even clicked on this 
same
> > symbol.
> >> 
> >> Next, I created a new FastTrack database with a different name 
and
> > copied
> >> over the appropriate files to the new DB folder. I enabled Local
> > Storage
> >> from the start.  All looked good.  Except that writes to the OI
> > field in the
> >> new DB would not save.  I tried the scan all symbols trick again,
> > and again
> >> it hung on the same symbol, certainly suggesting my FastTrack 
data
> > itself is
> >> corrupt on that one symbol. (Special updates with the FastTrack 
data
> >> designed to correct problems did not seem to change anything.)
> >> 
> >> My question is how do I get the FastTrack database to be local 
and
> > to allow
> >> writes to the OI field to stick? 
> >> My second question (as I assume I have to have the data in the
> > Amibroker
> >> folders and not just in the FastTrack folders for the AB DB to be
> > local), is
> >> how do I update the nightly downloads of new data which FastTrack
> > will stick
> >> in their own file/folder?
> >> 
> >> Thomasz, can you help?
> >> 
> >> Thanks,
> >> 
> >> Ken
> >>
> >
> >
> >
> > ------------------------------------
> >
> > Please note that this group is for discussion between users only.
> >
> > To get support from AmiBroker please send an e-mail directly to 
SUPPORT
> {at}
> > amibroker.com
> >
> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > http://www.amibroker.com/devlog/
> >
> > For other support material please check also:
> > http://www.amibroker.com/support.html
> > Yahoo! Groups Links
> >
> >
> >
> >
> > ------------------------------------
> >
> > Please note that this group is for discussion between users only.
> >
> > To get support from AmiBroker please send an e-mail directly to
> > SUPPORT {at} amibroker.com
> >
> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > http://www.amibroker.com/devlog/
> >
> > For other support material please check also:
> > http://www.amibroker.com/support.html
> > Yahoo! Groups Links
> >
> >
> >
> 
> ------------------------------------
> 
> Please note that this group is for discussion between users only.
> 
> To get support from AmiBroker please send an e-mail directly to
> SUPPORT {at} amibroker.com
> 
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
> 
> For other support material please check also:
> http://www.amibroker.com/support.html
> Yahoo! Groups Links
>



------------------------------------

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/