|
PureBytes Links
Trading Reference Links
|
On Nov 30, 9:55pm, Matt Proudfit wrote:
> I need an indicator that I can run in the workspace assistant that will
> cycle through all of the stocks in my portfolio (using a daily chart) and
> tell me the ones with no bars past a certain date (say 9/1/99, etc). I have
> some stocks in there that stopped trading due to mergers, failures,
> whatever. I want to weed them out and it takes forever going through
> manually.
Matt asked this question a few days ago. At the time, I
happened to be trying to go through and clean up the daily
data collection for my TS4 setup that uses EOD data
collected from Dial/Data. I collect most indexes, most
futures, and about 2000 stocks. I'd do more stocks if I
felt the system could handle it - and maybe it can now - up
until about 6 months ago, the Dial/Data server would get so
bogged down, that it would disconnect while attempting
auto-download, and downloading became a mostly manual
process of hitting "retry" after the download had flaked
out. The new server seems better, either that, or they
improved something in build 23 or 24 that improved the
reliability and speed of the download.
I'm pretty sure that Matt's question was regarding data
collected via the online real-time server, but thought I'd
offer a suggestion/two, if like me, you have a problem
keeping the Dial/Data downloads up-to-date. What I'm doing
now, occasionally, is to go to the downloader and ask it to
"print" the contents of a download directory to a "generic
text file" printer (that I defined by going to
Settings:Printers:Add Printer). I then wrote a Perl program
that does some consistency checking, and spits out a CSV
file that I can feed into Excel. I've included the Perl
script as an attachment, but must warn you that the code is
special-purpose, and like all Perl programs, is mostly "write
only".
BTW, this is the first time that I've tried out the Perl
port (found by following links at http://www.perl.com) for
Win9x, and so far it has worked rather well (this is the
Perl 5.005 port built by Active Software).
The script checks for stocks with the "download" option
set, whose last date is less than 30 days from the current
date. It also checks for futures whose last contract has
not been updated for 30 days (these probably should be
rolled to a new seies, unless the contract was
discontinued). Additionally, for stocks, the program spits
out a column indicating whether the stock is and ADR,
preferred, warrant, or "class" stock - making it easy to
sort on such criteria.
The real headache is that the downloader has no way to
accept a simple file containing a list of symbols that can
be used as input to a command thet says "unmark these for
download", or to say "add these symbols", or "roll these
futures to the next set of contracts, up to 6 months
forward". I almost had a few such maintenance tasks
automated with winbatch a couple of years ago, but ran out
of time and energy. I may try again over Xmas.
I haven't upgraded to TS2K - but just curious: is it better
in this reqard for data maintenance? Esp. end-of-day
data?
Does TS2K still support Dial/Data? Or only
historybank.com?
Does Omega still sell historical data CD's based on
Dial/Data's data?
What is the cost for historybank, or whatever else Omega is
recommending as an EOD source, if a trader wants to track
indexes, futures, and stocks?
X-Zm-Content-Name: filtdl.pl
Content-Type: text/plain ; name="filtdl.pl" ; charset=us-ascii
#!/usr/local/bin/perl -w
#
# Filter the output of the Omega Downloader, and look for symbols
# that no longer need to be downloaded.
#
# The output file is created in the downloader, by clicking on "Print"
# and then printing to a "Generic Text File" printer. If Test Filer
# printer is not set up, then go to the Settings:Printers:Add Printer
# selection and add one.
#
# The outfile has fixed format, Here's how a typical stock entry # looks:
# 1 2 3 4 5 6 7
#12345678901234567890123456789012345678901234567890123456789012345678901234567
#IBM International Busines * P F 01/05/1970 12/01/1999 D
#
# The "*" in column 48 indicates that this symbol will be downloaded.
# The dates are supposed to represent the earliest and latest dates
# in the file, but I've seen situations where thet information is
# incorrect.
use strict;
my @ignore_pats = (
qr/^$/,
qr/^Directory :/,
qr/^Printed :/,
qr/^Symbol/,
qr/^[ \t]+Page[ ]+[0-9]+\$/,
qr/^Prepared using Omega Downloader/
);
my ($a, $c, $now, $DAYS, %all_issues);
my ($t30daysago, $t30_day, $t30_month, $t30_year, $t30_date, $t30_yymm);
my ($pat, $symbol, $description, $date1, $date2);
my ($dl, $is_future, $is_index, $is_stock, $sym_type);
my (@desc, $last_word, $next2last);
my ($fut_sym, $fut_desc, %fut_sym_desc, $fy, $fm, $fut_yymm);
my (%fut_last_date, %fut_last_contract, $old_fut);
my ($last_dload_month, $last_dload_day, $last_dload_year, $last_dload_date);
my ($is_old, $issue);
sub dblq($) {
my($wrd) = @_;
if ($wrd !~ /^[a-zA-Z][a-zA-Z0-9]*$/) {
$wrd =~ s/"/""/g;
$wrd = '"' . $wrd . '"';
}
return $wrd;
}
BEGIN {
$[ = 1; # base arrays at 1!!!
# get today's date:
$now = time;
#and then calculate the date 30 days ago
$DAYS = 24*60*60;
$t30daysago = $now - 30 * $DAYS;
(undef,undef,undef,
$t30_day,$t30_month,$t30_year,
undef,undef,undef) = localtime($t30daysago);
$t30_month += 1;
$t30_year += 1900;
$t30_year += 100 if ($t30_year < 1920);
$t30_date = sprintf('%04d/%02d/%02d',
$t30_year, $t30_month, $t30_day);
$t30_yymm = substr($t30_date, 1, 7);
}
{
LINE: while (<>) {
chomp; # strip record separator
foreach $pat (@ignore_pats) { next LINE if ($_ =~ $pat); }
next LINE if (length() < 48);
$symbol = substr($_, 1, 10);
$symbol =~ s/\s*$//;
$description = substr($_, 17, 30);
$description =~ s/\s*$//;
$description =~ s/\s\s+/ /g;
$date1 = substr($_, 55, 10);
$date2 = substr($_, 66, 10);
$dl = substr($_, 48, 1);
$dl = '' if ($dl ne '*');
$is_future = ($symbol =~ /[a-zA-Z][a-zA-Z0-9 ][0-9]{4}/);
$c = substr($symbol,1,1);
$is_index = $c eq '*' || $c eq '#';
$is_stock = !($is_future || $is_index);
$sym_type = "STK";
$sym_type = "FUT" if ($is_future);
$sym_type = "INX" if ($is_index);
if ($is_stock) {
@desc = split(' ', $description);
$last_word = $desc[$#desc];
$last_word =~ tr/a-z/A-Z/;
$next2last = $#desc >= 2 ? $desc[$#desc-1] : '';
$next2last =~ tr/a-z/A-Z/;
$sym_type = "ADR" if ($last_word eq 'ADR');
$sym_type = "PFD" if ($last_word eq 'PFD');
$sym_type = "WT" if ($last_word eq 'WT');
if ($next2last eq "PFD" || $next2last eq "CL") {
$sym_type = $next2last;
}
}
if ($is_future) {
$fut_sym = substr($symbol, 1, 2);
$fut_desc = $description;
$fut_desc =~ s/ [0-9]{2}\/[0-9]{2}$//;
if (not exists $fut_sym_desc{$fut_sym}) {
$fut_sym_desc{$fut_sym} = $fut_desc;
}
$fy = int(substr($symbol,5,2)) + 1900;
$fy += 100 if ($fy < 1920);
$fm = int(substr($symbol,3,2));
if ($fm >= 1 and $fm <= 12) {
$fut_yymm = sprintf('%04d/%02d', $fy, $fm);
if ((not exists $fut_last_date{$fut_sym})
|| ($fut_yymm gt $fut_last_date{$fut_sym})) {
$fut_last_date{$fut_sym} = $fut_yymm;
$fut_last_contract{$fut_sym} = $symbol;
}
}
}
$is_old = '';
if ($dl eq '*') {
($last_dload_month,
$last_dload_day,
$last_dload_year) = split(/\//, $date2);
$last_dload_date = sprintf('%04d/%02d/%02d',
$last_dload_year, $last_dload_month, $last_dload_day);
$is_old = 'X' if ($last_dload_date lt $t30_date);
}
$all_issues{$symbol} = [$symbol, $description, $sym_type, $dl,
$is_old, '', $date1, $date2];
}
#
# Make a separate pass to tag the future symbols that
# may need to have new contracts added.
#
foreach $fut_sym ( keys %fut_last_date ) {
if ($fut_last_date{$fut_sym} lt $t30_yymm) {
# if the last contract for this future is some time ago
# then mark it with an 'R' indicating that this is a
# candidate for a possible "roll" to the newer contracts
$issue = $fut_last_contract{$fut_sym};
($all_issues{$issue})->[6] = 'R';
}
}
foreach $issue ( sort keys %all_issues ) {
($symbol, $description, $sym_type,
$dl, $is_old, $old_fut, $date1, $date2) = @{$all_issues{$issue}};
printf("%s,%s,%s,%s,%s,%s,%s,%s\n",
dblq($symbol), dblq($description), dblq($sym_type),
$dl, $is_old, $old_fut, $date1, $date2);
}
}
|