|
PureBytes Links
Trading Reference Links
|
Thanks guys. I was actually able to optimize SQL to get a decent performance that fits my needs. Again, my strategies are not high frequency.
I basically just created a SQL table with the AB required fields only. I also created an index specifically for AB's query. The historical data is inserted before the open and the RT data is inserted as trades occur.
Using the native ODBC plug-in, I am running an exploration for 200 tickers and 1000 daily bars in under 2 seconds. The same exploration using the local database takes about 1 second, therefore the performance is entirely satisfactory.
I can see how this is not the optimal solution for high frequency strategies that use intraday intervals or a watchlist with a great number of tickers. But it does work fine for more simple strategies.
--- In amibroker@xxxxxxxxxxxxxxx, "Yofa" <jtoth100@xxx> wrote:
>
> Hi,
>
> depending on what you want to do...
> You can use SQL but you must have in-depth knowlegde of the specific
> database sw.
> Logical and mostly physical database design is crutial to success. Both
> designs must be very efficent.
> Generally SQL database engine is used as a storage system. So it is bit
> overkill.
>
> Using SQL is handy if you want to store data and from this store you want to
> prepare/transform/export data to other processes. In such case it is very
> flexible.
> You CAN play back IB tick data at real time speed (~300ms snap shots) for a
> few tickers directly from SQL db.
> That should not be a problem with a good db design and on a good hw.
>
> Going beyond this limit is not for SQL!
>
> You must also consider SQL database resource needs. Disk and memory are
> needed to achive good performance.
> A binary file based storage system is very efficent but no really flexible.
> SQL is flexible but not efficient...
> You have to choose what you need..
>
> Regards,
>
> Y
>
> --------------------------------------------------
> From: "Tomasz Janeczko" <groups@xxx>
> Sent: Monday, May 18, 2009 9:37 PM
> To: <amibroker@xxxxxxxxxxxxxxx>
> Subject: Re: [amibroker] ODBC + DDE
>
> > SQL database & ODBC is wrong path for high-perf RT.
> >
> > ODBC is rather slow as it was never designed by Microsoft to be used as a
> > real-time streaming tool.
> > Also all standard SQL databases are way too slow for real time
> > applications.
> >
> > Only proprietary binary low-level dbs like AmiBroker own, that are very
> > low level and not SQL based
> > are able to offer right speed.
> >
> > As for interfaces - only direct APIs (like AmiBroker Dev Kit) that operate
> > directly
> > on binary data via direct pointers and no need for copying data and
> > without SQL overhead are fast enough.
> >
> > For commercial databases that offer better performance than SQL see
> > Berkeley DB
> > http://www.oracle.com/database/berkeley-db/db/index.html
> >
> > (Read their white papers that explain why all relational, sql-based
> > databases are simply too slow)
> >
> > Best regards,
> > Tomasz Janeczko
> > amibroker.com
> > ----- Original Message -----
> > From: "lucianomt" <lucianomt@xxx>
> > To: <amibroker@xxxxxxxxxxxxxxx>
> > Sent: Monday, May 18, 2009 6:30 PM
> > Subject: [amibroker] ODBC + DDE
> >
> >
> >>I am trying to set up real-time feeds on AB. I have a SQL database that
> >>contains all the necessary data. The only problem is the
> >>very high delay I get using the ODBC plug-in. To pull up about 1000 bars
> >>for 150 tickers takes approx. 10 minutes, which makes it
> >>impractical for real-time. (The same process using local database takes 1
> >>second.)
> >>
> >> So my first question is: am I doing something wrong? Is there any way to
> >> improve this performance significantly or is the ODBC
> >> solution inherently slow?
> >>
> >> If nothing can be done to improve ODBC, I thought I could upload the
> >> historical data into the local database and them use a
> >> different source, say DDE, to update the real-time quotes for the current
> >> day. Would that solution work?
> >>
> >> Please note that the historical data needs to be updated daily since it
> >> is a total-return series.
> >>
> >>
> >>
> >> ------------------------------------
> >>
> >> **** IMPORTANT PLEASE READ ****
> >> This group is for the discussion between users only.
> >> This is *NOT* technical support channel.
> >>
> >> TO GET TECHNICAL SUPPORT send an e-mail directly to
> >> SUPPORT {at} amibroker.com
> >>
> >> TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
> >> http://www.amibroker.com/feedback/
> >> (submissions sent via other channels won't be considered)
> >>
> >> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> >> http://www.amibroker.com/devlog/
> >>
> >> Yahoo! Groups Links
> >>
> >>
> >>
> >
> >
> >
> > ------------------------------------
> >
> > **** IMPORTANT PLEASE READ ****
> > This group is for the discussion between users only.
> > This is *NOT* technical support channel.
> >
> > TO GET TECHNICAL SUPPORT send an e-mail directly to
> > SUPPORT {at} amibroker.com
> >
> > TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
> > http://www.amibroker.com/feedback/
> > (submissions sent via other channels won't be considered)
> >
> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > http://www.amibroker.com/devlog/
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|