On Using CQG SP Data in TS

["Steven Buss" <sbuss@xxxxxxxxxxx>]

Recently purchased the full SP symbol data set (RTH and globex) from CQG
from 9/1993 to the present.  It's clear that I need to do a number of
transformations to the data to get it correct for use with TS and would like
to verify these changes with anyone else who has looked at this carefully.

After you read this note, you'll be asking:  why purchase data from CQG if
you have to go to all this trouble to convert the data for TS.  The answer
is that CQG was the only vendor I could find who had sp tick for both day
and night sessions back to the beginning of the night sessions.

I've got all the tick data in an Oracle database and I've already written
the code to make the changes listed below but before I flip the switch and
generate the new data set I was thinking that it would be nice to get
feedback on what I'm up to.  Your thoughts would be helpful.

There are several changes that need to be made and these are listed below:

-    I will adjust all timestamps by adding one minute to adjust for the cqg
timestamp meaning the open and the ts expectation that the timestamp is the
close.

-    The date stamp is incorrect for globex data for those trades from the
globex session open to 12pm midnight.  It appears that the date stamp for
these ticks is the date that the session data belongs to (i.e., the date
that the session CLOSES) not the date at which the trades actually take
place.  I talked to Laurie at CQG and she couldn't help me much.  TS is
expecting that the date stamp is the date that the trade actually takes
place.  The fix here is to subtract 1 day from the date of the globex trades
from the beginning of the globex session to 12pm midnight.

-    The timestamp on each data instance is Chicago time and has to be moved
forward one hour to Eastern time.

-    There are also various code fields on CQG tick data.  I'd like to
generate the new data set using only those instances that have the
appropriate codes.  I've listed the code types with the values they may have
below.  I've marked those code type values that I think should be included
with a "(+)".  I've marked those code type values that I think shouldn't be
included with a "(-)".

    Transaction Type:  B=Bid(-), A=Ask(-), T=Trade(+), S=Settle(+)
    Market Indicator:    F=Fast Market(+), N=Normal Market(+)
    Transaction Correction:    E=Exchange Delete(-), F=CQG Filter Delete(-),
                                                    I=Insert(+), N=Normal(+)

Thanks.

Steven Buss
sbuss@xxxxxxxxxxx
Walnut Creek, CA, USA