|
PureBytes Links
Trading Reference Links
|
Paul,
Initially I had been doing it manually. It hasn't been that bad (e.g.
copying and pasting the html output of individual walk forward
results into single unified file).
However, I have very recently begun experimenting with scripts to
automate the process and stitch the results back togeather.
I have a few ideas that I'm exploring, but am not yet able to
discuss 'till I've had a chance to further organize my thinking and
do a proof of concept.
It is still early in my efforts, and I have limited time (nights
after work and once the family is in bed), but first results look
promising.
In any event, I will gladly make available whatever I come up with
once it's reached a minimal level of functionality. So, if your spare
equiptment is not causing you too much hardship, it may be worth
hanging on to it a little longer.
Mike
--- In amibroker@xxxxxxxxxxxxxxx, "Paul Ho" <paul.tsho@xxx> wrote:
>
> Mike,
> Are you spreading your load over several machines manually, or
you're using
> programs/scripts to automate that process?
> I have a few old machines that I was thinking about throwing out,
may be I
> could utilise them if there was a practical way to share the load.
> /Paul.
>
>
> _____
>
> From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx]
On Behalf
> Of Mike
> Sent: Wednesday, 9 July 2008 6:26 PM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: [amibroker] Re: What is your Largest AFL file?
>
>
>
> Ken,
>
> For about a year now, I have been running a nightly exploration
over
> all symbols of AMEX, NASDAQ, NYSE combined. The script itself
applies
> logic to reject low liquidity stock, etc. and is applied to all
> stocks, rather than preprocessing a watchlist. The script is
> currently under 100 lines and completes in around 10 minutes (don't
> remember exactly, I stopped paying attention a long time ago).
>
> Optimizing that same script over 108 iterations takes anywhere from
> 9 - 13 hours depending on the machine/memory (strictly low end
> equiptment - Pentium 4's). Doing a walk forward of any meaningful
> length would take days, except that I split the load over multiple
> machines.
>
> In an earlier incarnation, I had been writing out to composite
> symbols in order to retain a sorted rank of PositionScore based on
> all symbols scanned, not just ones that resulted in trade signals.
I
> then made reference to those rankings within my custom backtest
code
> to filter out signals that would not have occured due to no limit
> order actually having been placed
> (http://finance.
> <http://finance.groups.yahoo.com/group/amibroker/message/114739>
> groups.yahoo.com/group/amibroker/message/114739).
>
> Ultimately, that proved to be prohibitively expensive (at least as
> implemented, Thomasz had an alternate approach that was much faster
> at the cost of altering the strategy logic), taking over 30 minutes
> for an exploration. Optimizations were frustrating and impractical,
> leading to my eventual purchase of many cheap low end machines to
> farm out the load. Walk forward would have been out of the question.
>
> Fortunately, I was able to change the money management aspect of
the
> system and move away from fixed number of positions. It's been
> trading profitably ever since :)
>
> Building on Herman's excellent suggestions, I would add that low
end
> machines (1.6 Ghz) can be had for under $150 these days (I got
lucky
> on eBay, paying $65 each for 7 of them!). Having a small farm
allows
> you to spread the load of a single walk forward (i.e. each machine
> handling a different time period) or to backtest/optimize multiple
> different versions of your strategy at the same time.
>
> Mike
>
> --- In amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
ps.com, "Ken
> Close" <ken45140@> wrote:
> >
> > A side question is this: what is the largest practical AFL file
> ever created
> > that runs and is used regularly?
> >
> > Obviously a hard question to answer (what is yours in terms of
> number of
> > lines or number of symbols operated on). We talk about memory and
> multiple
> > cores, and speed, etc, I will bet primarily in terms of
> optimization speed.
> > But what about just a large number of symbols and a large number
of
> > calculations and variables? That is what my question is aimed at.
> >
> > If you are wondering why this question, here is a brief
> background: all of
> > my recent postings have been because I am trying to work my way
> towards
> > implementation of what seems like a very large system. I want to
> calculate,
> > for each and every symbol, a ranking indicator or value really. I
> want to
> > use the "ordinal value" approach that I have recently been asking
> about.
> > Creating these ordinal values consumes a LOT of processing time
and
> > calculations etc. I would like to do this on 1000s of symbols
> (maybe the
> > entire database) but may have to settle for some subset, perhaps
> only in the
> > hundreds. But that is only the first step.
> >
> > After having these calculated values, I thought I would save them
> into the
> > OI field of each symbol. Then, separately, I would create an
> elaborate
> > rotational trading system, with a variety of parameters, and
which
> would use
> > the ordinal ranking parameter I had previously calculated and
saved
> into the
> > OI field. The rotational trading system, I am estimating, would
> take a lot
> > of statements and need to keep track of a lot of information, and
> that would
> > be before attempting any opimization of the trading parameters. I
> am
> > guessing that the rawbacktester mode is required.
> >
> > If separating the ranking calculation and saving it into the OI
> field is a
> > problem (because of the interplay between local storage vs plugin
> access
> > that I asked about in another message), then I could consider
> putting
> > everything (I mean everything) into one humongous AFL file.
> Calculate
> > ordinal rankings, keep them in their variables (by symbol) and
then
> go right
> > into a rotational trading routine, all in the same code file. I
> suspect it
> > would either overload memory, or would take days of continual
> running to
> > complete.
> >
> > That is why I asked what is the largest code file you have ever
> made. Maybe
> > my quest is a futile one, but I am still plowing along.
> >
> > Thanks for listening.
> >
> > Ken
> >
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|