|
PureBytes Links
Trading Reference Links
|
P/S What is meant by facilitating further rounds of sorting at end of
message, is that before running the batch file again I do not need to copy
the previous sorted end-file to the sorting directory and treat it as one of
text files to be further merged&sorted, for eg. Imagine remembering to copy
these files besides unzipping the right EODs for every round of the process,
verifying the right EODs are unzipped, that the sequence of the process are
strictly adhered, etc. In order to avoid merging repeated data, not to miss
any set of EODs, etc. It easily turned into a nightmare when you have to
start from the beginning after 20 rounds. My partial solution is to compile
by the weeks, verify for missing data only before going further by the
months, verify .... then year. At the end of the year, shall convert sorted
data into MetaStock format and back again to ACII text, to eliminate
repeated data. Can be one tedious processes but the consolation is that if
it is done on a daily basis, little by little like updating the present MS
database, it wouldn't be felt.
----- Original Message -----
From: "panda" <ckng001@xxxxxxxxxxxx>
To: <equismetastock@xxxxxxxxxxxxxxx>
Sent: Monday, March 21, 2005 8:24 PM
Subject: Re: [EquisMetaStock Group] Re: Merging ASCII data files
>
>
> Thank you Jose for early reply. (I think I can still trust my local data
> provider except for its world indices which it obtained from Yahoo I
> presume).
>
> Looks like the batch file will merge the original ASCII text files and
copy
> rpsort.com into zownload directory. Then it calls rpsort.com to sort the
> merged text file and put the results in another file, finally it deletes
the
> 2 working files and renames the end-results file.
>
> For each zipped EODs I have, it contained 25 smaller files. So, now I
could
> copy
> them all to a single mega-file for each year and update them daily like I
> update the MS database. Hmmm ... :)
>
> By the way, what are the switch >nul (other like /A, /B, \V can be found
at
> the DOS prompt help but not this) and the @ at ' @echo off '?
> Purpose for copying rpsort.com to zownload directory instead of putting it
> in directory beforehand?
>
> Hee Hee
>
> Best regards,
> ckng
>
> P/S Just try it. Great!
>
> But on further testing ... need to delete the header lines accumulated
>
manually(<TICKER>,<PER>,<DTYYYYMMDD>,<OPEN>,<HIGH>,<LOW>,<CLOSE>,<VOL>,<OPEN
> INT>) and some which attached to end of some lines 'cause the file from
> which it comes from do not have a carriage return at end.
> The copy command in batch file, cannot append new files to old ones... to
> facilitate further rounds of sorting.
>
> (For Info Purposes: for directory name of more than 1 word, try xxxxx~1 as
> name, since program
> run in DOS - eg. C:\My Documents becomes C:\MyDocu~1)
>
> ----- Original Message -----
> From: "Jose" <josesilva22@xxxxxxxxx>
> To: <equismetastock@xxxxxxxxxxxxxxx>
> Sent: Monday, March 21, 2005 1:10 AM
> Subject: [EquisMetaStock Group] Re: Merging ASCII data files
>
>
> >
> >
> > Panda, it really comes down to what is more important to you: time or
> > $.
> >
> > Personally I would avoid free EOD data like the plague, and get some
> > decent headache-free data such as http://www.premiumdata.net
> >
> > However, if you must sort & archive all that Yahoo EOD data, try
> > something like this batch file:
> >
> > ---8<--------------------------
> >
> > rem merges & sorts all *.txt into "update.txt", needs rpsort.com
> >
> > cls
> > copy /A *.txt c:\zownload\update.txt /V
> >
> > @echo off
> > copy /B rpsort.com c:\zownload\*.* /Y /V >nul
> > echo.
> > echo sorting update.txt mega-file...
> > echo.
> > c:\zownload\rpsort.com /q c:\zownload\update.txt >c:\zownload\update1.
> > srt
> > del c:\zownload\rpsort.com >nul
> > del c:\zownload\update.txt >nul
> > rename c:\zownload\update1.srt update.txt >nul
> > echo.
> > echo ... done!
> > echo.
> >
> > ---8<--------------------------
> >
> > You can find RPSort.com if you do a search on Google.
> >
> >
> > jose '-)
> > http://www.metastocktools.com
> >
> >
> >
> > --- In equismetastock@xxxxxxxxxxxxxxx, "panda" <ckng001@xxxx> wrote:
> > >
> > > Hi all,
> > >
> > > I have a rare occasion where my database for one particular stocks
> > contained
> > > an unusual spike on one of its bar. Upon re-updating, the spike
> > > disappeared. Moreover, some of my raw EOD data files contained
> > errors which
> > > I manually corrected (errors occurred in world indices where it's
> > not as
> > > critical - free EODs, so what do you expect!). For these two
> > reasons, I am
> > > now keeping all my EOD files that I used to update my MS database
> > daily,
> > > just in case.
> > >
> > > So as each trading days go by, the no. of EOD files grows bigger and
> > bigger.
> > > So now I am looking at merging all these EOD files by the year
> > (2004, 2005,
> > > etc.) before zipping them. Moreover, I would like to sort the data
> > > according to stocks (instead of dates in the original ASCII files)
> > in the
> > > merging process. All this will simplify and quicken the updating
> > process
> > > with downloader if ever I need to re-update. So, anybody got an
> > efficient
> > > way of going about this? The right merging&sorting software, etc ..
> > . (Each
> > > EOD file consists of 25 ACII text files). (Excel could sort data
> > but saving
> > > them back into ASCII files as per original format (with commas) will
> > be a
> > > problem, opening each individual files for each EOD and copy&paste
> > will be
> > > cumbersome, saving the MS database will not do as some of the stocks
> > which I
> > > seldom look into may contained errors).
> > >
> > > Thank you for your time in reading and answering my message.
> > >
> > > Best regards,
> > > ckng
> >
> >
> >
> >
> >
> >
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
> >
> >
> >
>
>
>
>
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
>
------------------------ Yahoo! Groups Sponsor --------------------~-->
In low income neighborhoods, 84% do not own computers.
At Network for Good, help bridge the Digital Divide!
http://us.click.yahoo.com/EpW3eD/3MnJAA/cosFAA/BefplB/TM
--------------------------------------------------------------------~->
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/equismetastock/
<*> To unsubscribe from this group, send an email to:
equismetastock-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|