[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [amibroker] Re: "SLOW" backfill and delay with IQFeed



PureBytes Links

Trading Reference Links

Hello,

I did the following benchmark for you (that you can also duplicate on your 
end to verify if you are getting similar results).

Reproduction steps:
1. File->New->Database
2. Create new database
3. Select Data Source from the combo (DTN IQFeed or eSignal or other)
4. Enter 100000 into "Number of bars
5. Select Base time interval: 1 minute and click OK button
6. Wait until green light appears - so it is connected to the data source
7. Add MSFT (enter it in ticker box to display the chart)

TIME MEASUREMENTS as of Nov 5, 2008, 8:44PM ET

Data source: IQFeed, symbol: MSFT, interval: 1 minute,  74934 bars arrived 
TOTAL BACKFILL TIME: 15.2 sec

Backfill rate: 4929 bars per second

Data source: eSignal , symbol: MSFT, interval: 1 minute, 59258 bars arrived 
TOTAL BACKFILL TIME: 5.5 sec

Backfill rate: 10774 bars per second

You should be getting similar results when you replicate conditions mentioned above.

Please note that these measurements were done AFTER MARKET CLOSE.
This makes significant difference as compared to backfills during market hours
when they are noticeably slower. 
I would need to do measurements tommorrow, but it seems that at least for IQFeed
the difference between pre/after hours and during market hours can be as large
as 5-10 times.


Best regards,
Tomasz Janeczko
amibroker.com
----- Original Message ----- 
From: "Tomasz Janeczko" <groups@xxxxxxxxxxxxx>
To: <amibroker@xxxxxxxxxxxxxxx>
Sent: Thursday, November 06, 2008 2:06 AM
Subject: Re: [amibroker] Re: "SLOW" backfill and delay with IQFeed


> Hello,
> 
> I wrote you everything already:
> a)
> IQFeed LIMITS the speed of your transfer (as everyone else).
> They have tens of thousand subscribers and they need to do that
> to make sure that one user does not take all capacity.
> 
> b) the speed of backfill is
> SOLELY dependent on IQFeed server response. 
> 
> So please complain to IQFeed support staff.
> 
> Best regards,
> Tomasz Janeczko
> amibroker.com
> ----- Original Message ----- 
> From: "przemyslawuk" <przemyslawuk@xxxxxxxxxxx>
> To: <amibroker@xxxxxxxxxxxxxxx>
> Sent: Thursday, November 06, 2008 12:27 AM
> Subject: [amibroker] Re: "SLOW" backfill and delay with IQFeed
> 
> 
>> Hi again,
>> My observations confirm what monitorit says.
>> 
>> Doing the calculations from other end- database settings :
>> 1min/1100bars/34kb per symbol.
>> Backfill of 60 symbols took me 47min (before the market open and I was
>> not runing AB the day before).
>> 
>> 60symbols * 34kb per symbol -> ~2Mb  during 47min -> gives avarege
>> backfill transfer of 0.7kb/s   !!!!!!!!!!!!!!!!!!
>> 
>> WHERE IS THE BOTTLENECK?
>> 
>> Obviously streaming large number of symbols and runing scans whole day
>> with a 0.7kb/s transfer has to cause delays. And I noticed that if I
>> only use Quotetracker the delay is not that large.
>> 
>> If I can download files from internet at 500kb/s the backfill should
>> have similar transfer rate.
>> 
>> Can other users of AB confirm the numbers I get by running similar
>> test on their database?
>> 
>> 
>> 
>> --- In amibroker@xxxxxxxxxxxxxxx, "monitorit" <monitorit@xxx> wrote:
>>>
>>> Tomasz, I use qpFeed not IQfeed but have had a problem forever that 
>>> is somewhat the same.  There is NO problem when just updating a few 
>>> symbols continuously-but- if I try to keep about 75-100 symbols 
>>> updating, the quotes come in slower and slower.  The people at the 
>>> quote provider say they get no complaints and there is no problems 
>>> sending out the quotes.  Can you try to set up a 1 minute db with 
>>> about 100 tickers, the layout has 5 windows all symbol-linked (1min, 
>>> 2min, 5min, 15min and daily).  Use any simple indicator with the 
>>> price (ie 10 bar MA).  Run any simple AA to cause all 100 tickers to 
>>> update.  Then see how long the symbols take to get up to date and 
>>> more important, come back a few hours later and see the time of the 
>>> last bar compared to the time on your clock.. has it fallen behind?  
>>> You can just leave the original symbol displayed- no need to change 
>>> symbols after the first change after running the AA.  I know we have 
>>> discussed this before- but I wonder if you have done my little 
>>> experiment yourself?
>>> Thanks again and I remain convinced AB is the best by far.
>>> Dan
>>> 
>> 
>> 
>> 
>> 
>> ------------------------------------
>> 
>> **** IMPORTANT ****
>> This group is for the discussion between users only.
>> This is *NOT* technical support channel.
>> 
>> *********************
>> TO GET TECHNICAL SUPPORT from AmiBroker please send an e-mail directly to 
>> SUPPORT {at} amibroker.com
>> *********************
>> 
>> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
>> http://www.amibroker.com/devlog/
>> 
>> For other support material please check also:
>> http://www.amibroker.com/support.html
>> 
>> *********************************
>> Yahoo! Groups Links
>> 
>> 
>> 
> 
> ------------------------------------
> 
> **** IMPORTANT ****
> This group is for the discussion between users only.
> This is *NOT* technical support channel.
> 
> *********************
> TO GET TECHNICAL SUPPORT from AmiBroker please send an e-mail directly to 
> SUPPORT {at} amibroker.com
> *********************
> 
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
> 
> For other support material please check also:
> http://www.amibroker.com/support.html
> 
> *********************************
> Yahoo! Groups Links
> 
> 
> 

------------------------------------

**** IMPORTANT ****
This group is for the discussion between users only.
This is *NOT* technical support channel.

*********************
TO GET TECHNICAL SUPPORT from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com
*********************

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html

*********************************
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/