|
PureBytes Links
Trading Reference Links
|
Excellent, excellent, excellent! Tomasz your post gives me more
information than I could have reasonably expected.
I do have a couple of followup questions for clarity. Since IB's
streaming data is time based (0.2 to 0.3 second snapshots) and AB
counts each IB snapshot as one tick (i.e. one trade) when the base
interval is set to tick, then the most ticks IB allows AB to count is
5 ticks per second. Every AB chart tick bar is assigned a
timestamp. Since the current timestamp resolution is 5 seconds, then
each tick bar would have a 5 second timestamp assigned to it.
Suppose AB counted 25 ticks during the first 5 seconds of plotting a
chart. Let's also assume for simplicity that those 25 ticks occur
exactly between two 5 second timestamps.
Here's my follow up questions:
1. If I am plotting a 10t chart (assuming tick mode and all other AB
settings are correct), would AB display three 10t chart bars (two
historical bars of 10 ticks and one current bar which is partially
finished at 5 ticks) for those 25 ticks where the two historical 10t
chart bars would have identical 5 second resolution timestamps and
the current bar would receive a future timestamp once it completes?
2. If not how many tick bars would AB display on a 10t chart for
those 25 ticks?
Thank you very much for your previous reply, it gave myself and
probably many others an enhanced understanding of the IB data feed.
Regards,
David
--- In amibroker@xxxxxxxxxxxxxxx, "Tomasz Janeczko \(groups\)"
<groups@xxx> wrote:
>
> >
> >>From your documents and the forum post you reference, I was
unable to
> > make the interpretive leap on my own that "InvestorRT's chart is
> > wrong because it counts duplicate events as separate ticks".
Stupid
> > me. The subject of "realtime" IB data is complicated and there
is
> > much mis-information on the internet that confuses the issue.
>
> Actually what I meant that any *tick* chart using IB feed is wrong,
because
> IB does not send ticks.
> In fact there is no "right" or "wrong" way of counting events from
IB feed
> because IB did not really reveal ANY documentation covering their
internal
> tick processing and 0.3 sec snapshot creation and all knowledge we
have
> is based on guesses, experimentation, manual comparison from
different feeds
> and alike "investigation".
> This is far from scientific approach and it is closer to alchemy.
> Despite numerous requests from many developers IB does not really
care
> to document their feed in detail and developers rely on their own
experimentation.
> What I found by analysing TWS API source is that the API sends
duplicate
> TWO events (tickPrice and tickVolume) if price changes and only ONE
event
> (tickVolume) if volume changes during 0.3 sec period. Sometimes it
can send
> three events (tickPrice and two tickVolume events). If this happens
during 0.3
> second period you don't know whenever it was one, two, ten or
hundreds of trades (ticks).
> All you know that price or volume or both changed.
> Now what is a "proper" way of counting ???
> AmiBroker does NOT count tickPrice and tickVolume as TWO ticks if
they come from single price change.
> Other softwares may count it as two.
> Which is correct? Neither one or second. Both are wrong.
> Why? Because during 0.3 second there could be 1,2, 3, 10, 50 or 100
trades.
> No one knows that looking at IB feed stream. During news release
(as FED rate decision)
> there can be hundreds of trades flowing in sub-second periods on
Eminis,
> yet you will see only one IB-tick per 0.3 sec.
>
> > 2. Does anyone know why AB would be aggregating ticks to a time
slice
> > that is a multiple of five? Are there settings to control this?
> If base time interval is set to TICK, AB does not aggregate ticks
to any time slices
> other than IB own 0.3 sec minimum snapshot interval.
> Please note that if you request *BACKFILL* then IB limits backfill
resolution to 1-second bars.
>
> As far as STREAMING data is considered
> if base time interval is set to tick, one 0.3 second update (if
there is any change
> in price or volume) is counted as one tick. Timestamping does not
matter.
> Note that current timestamp resolution is 5 second, but it does not
mean
> any aggregation as there will be usually MANY ticks with same
timestamp.
> The same would occur with 1 second resolution as there are MANY
ticks
> during one second and MANY data points would share the same
timestamp.
> So timestamping does not matter and is irrelevant to tick mode.
> The only thing where it matters is creation of sub-minute TIME
based bars
> and AB currently allows 5-second and 15 second bars.
> The resolution of timestamp will be increased to sub-second
sometime in the future
> but this is not relevant to tick charts. It is only needed to be
able to create
> N-second charts where N is any natural number like (2-second, 7-
second, 9-second charts).
>
>
> > 3. Are AB tick charts and tick data collection dependant on the
chart
> > refresh interval? In other words would a 10t chart with a chart
> > refresh interval of 0 be different than a 10t chart with a
refresh
> > interval of 3 seconds?
> >
>
> No, ticks are collected one by one independently without any
reference to chart refresh interval.
> Making chart refresh interval longer will simply update chart less
often so for example
> you will see 10 new ticks at once after 3 seconds on actively
traded symbol.
>
>
> Best regards,
> Tomasz Janeczko
> amibroker.com
>
> ----- Original Message -----
> From: "dbwyatt_1999" <dbw451@xxx>
> To: <amibroker@xxxxxxxxxxxxxxx>
> Sent: Wednesday, December 26, 2007 4:34 PM
> Subject: [amibroker] Re: AB tick bar analysis
>
>
> >
> > I do not know whether IRT's "chart is wrong" is a fact or your
> > deduction based on the fact that IRT tick charts are different
than
> > AB tick charts. I cannot assume it's a fact since you also said
that
> > IB data is not real tick data (probably 1000s of times), and it's
up
> > to each software developer to interpret the IB data stream to
create
> > tick charts. This explains to me why there is a difference
between
> > IRT and AB charts and answers my first question in my initial
post.
> > I do not have enough knowledge nor even care to evaluate whether
> > IRT's implementation interpretation is wrong or AB's
implementation
> > interpretation is right.
> >
> > The docs and forum posts do not explain to me why my AB tick
chart
> > bars all have timestamps that are multiples of 5 seconds. John's
> > post gave me an explanation, but you say "No, it does not make
ANY
> > sense" and accuse me again of not reading docs. Since tick mode
does
> > not use timestamps, then I do not understand why all my AB 10t
bars
> > have timestamps that are multiples of 5 seconds. I either missed
> > something in the documentation or I'm unable to make an
interpretive
> > leap between the documentation and my question.
> >
> > I initially thought maybe I have something in AB setup
incorrectly,
> > so that's why I asked the group a few questions in my initial
post.
> > For reference, here are my initial questions again:
> >
> > 1. Does anyone know why AB would produce less tick bars than
> > Investor/RT using the same data feed? ANSWERED - thank you.
> > 2. Does anyone know why AB would be aggregating ticks to a time
slice
> > that is a multiple of five? Are there settings to control this?
> > 3. Are AB tick charts and tick data collection dependant on the
chart
> > refresh interval? In other words would a 10t chart with a chart
> > refresh interval of 0 be different than a 10t chart with a
refresh
> > interval of 3 seconds?
> >
> > Regards,
> >
> > David
> >
> >
> >
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "Tomasz Janeczko" <groups@>
> > wrote:
> >>
> >> No, it does not make ANY sense. You guys do not read docs.
> >> Timestamping has nothing to do with tick mode. Tick mode and any
n-
> > tick chart does NOT use timestamps at all.
> >> They are completely irrelevant. The reason Please read the
User's
> > Guide
> >> http://www.amibroker.com/guide/h_ib.html
> >>
> >> It clearly explains that Interactive Brokers is NOT TICK FEED!
> >> 4. IB TWS streaming data are NOT tick-by-tick, but rather 0.2-
0.3
> > second snapshots, read this for details
> >> http://www.interactivebrokers.com/cgi-bin/discus/board-auth.pl?
> > file=/2/37364.html
> >>
> >> See also my other reply
> > http://finance.groups.yahoo.com/group/amibroker/message/118232
> >> for details and explanation why InvestorRT chart is wrong because
> >> it counts duplicate events as separate ticks. Timestamps would
NOT
> > change that.
> >>
> >> For true tick charts you need to use tick-by-tick feed which is
> > eSignal or IQFeed.
> >>
> >> Best regards,
> >> Tomasz Janeczko
> >> amibroker.com
> >> ----- Original Message -----
> >> From: "dbwyatt_1999" <dbw451@>
> >> To: <amibroker@xxxxxxxxxxxxxxx>
> >> Sent: Wednesday, December 26, 2007 4:23 AM
> >> Subject: [amibroker] Re: AB tick bar analysis
> >>
> >>
> >> > Thanks John, that makes sense. Assuming the realtime feed in
> >> > AmiBroker Pro is the same 5 second timestamp scheme for
> > aggregating
> >> > realtime tick data as the ASCII tick data import process, then
> > that
> >> > explains the 5 second timestamps I'm seeing on my 10t bars.
If
> > that
> >> > is the case, then I guess AB really cannot handle small
timeframe
> >> > tick charts used by scalpers. The tick bars created using a
> > minimum
> >> > resolution of 5 second aggregated data will create inaccurate
> > small
> >> > timeframe tick bars. This explains the tick chart differences
> > I'm
> >> > seeing when compared the other charting package.
> >> >
> >> > Also, thanks to David C. for previously posting his tick data
> >> > importing explanation.
> >> >
> >> > Regards,
> >> >
> >> > David
> >> >
> >> >
> >> > --- In amibroker@xxxxxxxxxxxxxxx, "john_dxd_smith"
> >> > <john_dxd_smith@> wrote:
> >> >>
> >> >>
> >> >> Let's hope for 1-second resolution.
> >> >>
> >> >> ------------------------------------------------------------
> >> >> http://finance.groups.yahoo.com/group/amibroker/message/97297
> >> >>
> >> >> From: "David Clayworth" <dclayw@>
> >> >> Date: Thu May 25, 2006 8:28 pm
> >> >> Subject: RE: [amibroker] Time stamp - ASCII Tick Data Import
> >> >>
> >> >> I asked this question of AB support some time ago. This is the
> >> >> response I got:
> >> >>
> >> >> "This is due to the fact that AmiBroker uses its own 32-bit
> >> > datestamp
> >> >> that has resolution of 5 seconds but spans years of 1900-2155.
> >> >> Standard C-language 32-bit timestamp that has resolution of 1-
> >> > second is
> >> >> limited to dates from 1970-2023 and therefore can not be used
> >> >> (year range is to narrow).
> >> >>
> >> >> However this does not really affect anything as all ticks are
> >> > imported,
> >> >> order of ticks is maintaned and all charts look correct and
all
> >> >> analysis is accurate."
> >> >>
> >> >> Note that I only have the standard version of AB so I am
unsure
> > if
> >> > it
> >> >> is different for the professional version.
> >> >>
> >> >> ------------------------------------------------------------
> >> >>
> >> >> --- In amibroker@xxxxxxxxxxxxxxx, "dbwyatt_1999" <dbw451@>
wrote:
> >> >> >
> >> >> > Thanks Tomasz. I am not using the ASCII importer. I'm
using
> > the
> >> > IB
> >> >> > plugin to receive data from IB TWS in realtime. For my
> > analysis,
> >> > I'm
> >> >> > running three programs on the same computer: IB TWS,
AmiBroker
> >> > Pro,
> >> >> > and Investor/RT. AB and IRT receive the same data stream
from
> > IB
> >> > TWS
> >> >> > in real-time at the same time. I took a detailed look at
the
> >> > price
> >> >> > bars created on a 10 tick chart for a 10 minute period in
both
> > AB
> >> > and
> >> >> > IRT. AB created 14 10t bars and IRT created 22 10t bars
for
> > the
> >> > same
> >> >> > 10 minute period using the exact same IB data feed. Total
CPU
> >> >> > utilization stays under 10% for the entire test period.
The
> >> > total
> >> >> > volumes for the 10 minute period are approximately the same
> > for
> >> > both
> >> >> > AB and IRT 10t charts. The difference in the number of 10t
> > bars
> >> >> > created between the two charting packages is what concerns
> > me. I
> >> >> > suspect that I do not have something setup correctly with
AB
> >> > because
> >> >> > all the AB tick bars have a timestamp that are a multiple
of 5
> >> >> > seconds.
> >> >> >
> >> >> > Is there some AB setting that is forcing the timestamps to
be
> >> >> > multiples of 5 seconds for a 10 tick chart created in
realtime
> >> > from
> >> >> > the IB TWS data feed?
> >> >> >
> >> >> > Thanks,
> >> >> >
> >> >> > David
> >> >> >
> >> >> >
> >> >> > --- In amibroker@xxxxxxxxxxxxxxx, "Tomasz Janeczko"
<groups@>
> >> >> > wrote:
> >> >> > >
> >> >> > > Hello,
> >> >> > >
> >> >> > > If you are using ASCII importer and want to import tick
data
> >> >> > > you must use special mode, because ticks are special in
that
> >> > meaning
> >> >> > > that ticks may have duplicate timestamps. For that reason
> > you
> >> > need
> >> >> > > to specify
> >> >> > >
> >> >> > > $TICKMODE 1
> >> >> > >
> >> >> > > in the ASCII importer definition.
> >> >> > >
> >> >> > > See the documentation of ASCII importer - it has very
> > detailed
> >> > docs
> >> >> > > in the User's Guide.
> >> >> > >
> >> >> > >
> >> >> > > Best regards,
> >> >> > > Tomasz Janeczko
> >> >> > > amibroker.com
> >> >> > > ----- Original Message -----
> >> >> > > From: "dbwyatt_1999" <dbw451@>
> >> >> > > To: <amibroker@xxxxxxxxxxxxxxx>
> >> >> > > Sent: Monday, December 24, 2007 10:44 PM
> >> >> > > Subject: [amibroker] Re: AB tick bar analysis
> >> >> > >
> >> >> > >
> >> >> > > > Thanks Barry, you do have a point that IB data does not
> > have
> >> > a
> >> >> > > > timestamp which would make comparing different captures
> >> >> > difficult.
> >> >> > > > The Scottrade DJI tick data looks like it's 2 second
> >> > snapshots.
> >> >> > > >
> >> >> > > > I'm not so much concerned about having exact
timestamps,
> > but
> >> > the
> >> >> > > > number of tick bars for a time period for the exact
same
> >> > datafeed
> >> >> > > > should be very close regardless of the timestamps. I
mean
> > 10
> >> >> > ticks
> >> >> > > > is 10 trades, whether that took 0.23 seconds or 35
> > seconds.
> >> > The
> >> >> > > > number of trades (i.e. number of bars) should be
similar
> >> > between
> >> >> > > > different charting packages. They may not match up
> > depending
> >> > on
> >> >> > > > where the tick counting starts, but overall the total
> > number
> >> > of
> >> >> > bars
> >> >> > > > should be very similar between all charts with the same
> > tick
> >> >> > > > timeframe. If 1000 trades happened in a given time
> > period,
> >> > then
> >> >> > on a
> >> >> > > > 10 tick chart there should be close to 100 bars.
> >> >> > > >
> >> >> > > > Regards,
> >> >> > > >
> >> >> > > > David
> >> >> > > >
> >> >> > > > --- In amibroker@xxxxxxxxxxxxxxx, "Barry Scarborough"
> >> >> > <razzbarry@>
> >> >> > > > wrote:
> >> >> > > >>
> >> >> > > >> I think the reason is that IB does not provide a time
> > stamp
> >> > with
> >> >> > > > the
> >> >> > > >> data it sends and AB has to use the computer clock. If
> > that
> >> > is
> >> >> > the
> >> >> > > >> case the data never will be in sync. That is what I
have
> >> >> > noticed.
> >> >> > > > My
> >> >> > > >> last bar close on one minute intraday charts is never
the
> >> > same
> >> >> > as
> >> >> > > >> Scottrade or the EOD database from Yahoo. IB sends
data
> >> > before
> >> >> > the
> >> >> > > >> open and after the close and that can get shifted into
> > the
> >> > AB
> >> >> > > > charts
> >> >> > > >> depending on your computer clock. Try looking at a few
> > bars
> >> >> > either
> >> >> > > >> way and see if the open and close is the same on them.
> > But
> >> > then
> >> >> > I
> >> >> > > >> guess when you are looking at really short term data
even
> >> >> > different
> >> >> > > >> data feeds may not be in sync.
> >> >> > > >>
> >> >> > > >> I dumped the last 100 bars of DJI tick data from
> > Scottrade.
> >> > If
> >> >> > you
> >> >> > > >> have access to IB INDU tick data you could compare
them.
> > I
> >> > can
> >> >> > send
> >> >> > > >> the file if that would help.
> >> >> > > >>
> >> >> > > >> Barry
> >> >> > > >>
> >> >> >
> >> >>
> >> >
> >> >
> >> >
> >> >
> >> > Please note that this group is for discussion between users
only.
> >> >
> >> > To get support from AmiBroker please send an e-mail directly
to
> >> > SUPPORT {at} amibroker.com
> >> >
> >> > For NEW RELEASE ANNOUNCEMENTS and other news always check
DEVLOG:
> >> > http://www.amibroker.com/devlog/
> >> >
> >> > For other support material please check also:
> >> > http://www.amibroker.com/support.html
> >> >
> >> > Yahoo! Groups Links
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
> >
> >
> >
> >
> > Please note that this group is for discussion between users only.
> >
> > To get support from AmiBroker please send an e-mail directly to
> > SUPPORT {at} amibroker.com
> >
> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > http://www.amibroker.com/devlog/
> >
> > For other support material please check also:
> > http://www.amibroker.com/support.html
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
> >
>
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|