|
PureBytes Links
Trading Reference Links
|
Many of us know that CME makes it's GLOBEX T&S files available in it's
ftp site. Although the data format is a bit convoluted, it's possible to
extract the data in a fairly automated format, like many of you
undoubtedly do. I use IB's data for realtime trading, but in the
weekend, I purge out the weeks data from GS and replace it with the CME
data, ensuring I have cleanest, "official" data, tick by tick, with no
gaps around. Great for backtesting. And I bet this is much cleaner data,
with most accurate timestamps than you can have from any "realtime"
provider, like esignal et. al. Also, a great way to fill gaps at the end
of the day, should you want to.
And I do this with tools available for free, in a completely automated
fashion. Well, you have to double click on a file to run it, but that's
about it. xpo files according to your specification would be created
automatically. You can create daily data with any arbitrary session
times you specify. All bad ticks / busted trades are filtered out.
You can use this utility to create tick, minute & daily data for any
globex symbol, with arbitrary daily session times (even multiple session
times - like day only, all session, whatever)
The script does the following - gets inputs from command line &
inputs.txt, unzips the cme files, processes them to create data files as
per specifications in inputs.txt, feeds the ascii files to hypertools
and the .xpo is ready. All you do is import the .xpo files in GS.
It's taken me a fair amount of time to develop, and more importantly,
debug. Here's my reasons for providing it at no cost:
1) This list has given me a lot. Just a way to say thank you.
2) I do hope that if you use it, and find it useful, you would make a
donation (whatever the tool's worth to you) to appreciate my work. If
you need help setting it up, or need modifications, I can do that as
well. If you do decide to send a donation, paypal to payment@xxxxxxxxxxx
Thanks!
3) Donations of old GLOBEX T&S data files are very much appreciated. Ask
me what time range I need.
4) Although I am trying to make most of my money through trading, I keep
doing a fair amount of coding work. For reasonable compensation, I can
develop stuff for you.
**** WARNING : Some setup required. Expect to spend some time to set
this up. But once this is done, no time is required on a daily basis to
use it. It's fully automated. You can use / modify this in any way you
see fit, but you may not use this commercially. I retain copyright for
my work ****
That being out of the way, here's how to get the globex files.. automated
------------------------------------------------------------------------
I use the wget utility from a unix box to do this for me, from a
automated cron job. wget exists for windows as well. Just search google
for wget windows. The crontab entry is:
40 20 * * * /usr/bin/wget --passive-ftp -N -nd
--directory-prefix=/big/trading/data/cme/download/
--append-output=/big/trading/data/cme/download/abhijit_cron_wget.log
ftp://ftp.cme.com/pub/gtime/*.zip
Once this is setup, I never have to worry about it, the files are
downloaded automatically. You have to change the directory names, of
course. Not only that, as CME keeps updating this file (yes, they do) to
put later versions with some ticks marked as bad / bust trades etc.,
they would come in automatically, based on time stamp. You might want to
try windows scheduler / something else to do this at a set time everyday.
But you can forgo all of the above and just use any ftp client to get it
from ftp.cme.com/pub/gtime
How to setup
--------------
You need perl installed. I use activestate.com's perl distribution. It's
free. But any perl distro should do, I think.
You need hypertools installed. From hyperserver.com. It's free. After
installing, "import" the .zip files (3 of them) in HyperTools, from
C:\globexdata\bin (read below).
Unzip the attached .zip file to your C drive, keeping the paths intact.
You would have a "globexdata" directory, with some subdirectories inside
it, along with the code & sample input file. All these directories etc.
can be changed, if you need to. But you need to tinker with code / input
file / other tools.
This script takes some input from the commandline, explained in the perl
script itself. Most other inputs are taken from the file, yes you
guessed it, inputs.txt.
**** Review the inputs.txt file. Keep a backup, but you NEED to modify
it to specify what data you want ****
After modifying the inputs.txt file, and assuming you have the globex
.zip files saved in c:\globexdata\idir, you are all set to go.
Run it like this
perl futI.pl
Or just run the futI.pl file from windows explorer by double clicking on it.
The optional commandline switches are
-d yymmdd || yymmdd-yymmdd || ALL || AUTO
if omitted, default is all files that need to be processed, based
on timestamp (same as AUTO)
-i source_dir # if omitted, default current dir
-o output_dir # if omitted, default current dir
-i & -o can also be specified in the inputs.txt file, and indeed are.
Commandline switches overrides input.txt
On to the inputs.txt file provided. Go to the end, you would see things like
sym=EC_0412_EURZ4_1700_1600
sym=ES_0412_ESZ4_830_1515_ESZ4+1_1530_1515
It means extract data for EUR (EC) for 0412 expiry, and corresponding GS
symbol is EURZ4. Daily session starts at 1700, ending at 1600 next day.
For ES, create 2 sets of data for GS; ESZ4 for daily, ESZ4+1 for all
sess. You can, and should modify all these sym= lines to get what you
want / need. The 0412 specifies the month & year of contract. If you
want the H05 contract as well, you would add this line
sym=ES_0503_ESH5_830_1515_ESH5+1_1530_1515
if all you need is day see, you would instead add
sym=ES_0503_ESH5_830_1515
Keep this lines down to what you need. The more lines, the more time it
takes to extract / convert data. Also,
sym=ES_0503_ESH5_830_1515_ESH5+1_1530_1515
takes double the time to execute than
sym=ES_0503_ESH5_830_1515
As you can see, you only need to modify this file during contract rollovers.
**** important line in inputs.txt ****
makexpo=tmd
if you don't need tick data, change it to
makexpo=md
results in a significant time saving.
Hope you all find it useful. Thanks,
Abhijit
If you didn't receive the attached .zip file, email me at
adey@xxxxxxxxxx and I would send it to you.
Attachment:
globexdata.zip
Description: Zip compressed data
|