|
PureBytes Links
Trading Reference Links
|
Mike:
Thanks for sharing your experience and the idea of a low cost computer
"farm" to cut down on time to run long explorations, backtests,
optimizations. Interesting idea for sure. I am quite a ways from where you
are, but will review the code in the message you referenced for calibration
on more complex logic and execution steps than I am used to. I appreciate
your sharing.
Ken
-----Original Message-----
From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf
Of Mike
Sent: Wednesday, July 09, 2008 4:26 AM
To: amibroker@xxxxxxxxxxxxxxx
Subject: [amibroker] Re: What is your Largest AFL file?
Ken,
For about a year now, I have been running a nightly exploration over all
symbols of AMEX, NASDAQ, NYSE combined. The script itself applies logic to
reject low liquidity stock, etc. and is applied to all stocks, rather than
preprocessing a watchlist. The script is currently under 100 lines and
completes in around 10 minutes (don't remember exactly, I stopped paying
attention a long time ago).
Optimizing that same script over 108 iterations takes anywhere from
9 - 13 hours depending on the machine/memory (strictly low end equiptment -
Pentium 4's). Doing a walk forward of any meaningful length would take days,
except that I split the load over multiple machines.
In an earlier incarnation, I had been writing out to composite symbols in
order to retain a sorted rank of PositionScore based on all symbols scanned,
not just ones that resulted in trade signals. I then made reference to those
rankings within my custom backtest code to filter out signals that would not
have occured due to no limit order actually having been placed
(http://finance.groups.yahoo.com/group/amibroker/message/114739).
Ultimately, that proved to be prohibitively expensive (at least as
implemented, Thomasz had an alternate approach that was much faster at the
cost of altering the strategy logic), taking over 30 minutes for an
exploration. Optimizations were frustrating and impractical, leading to my
eventual purchase of many cheap low end machines to farm out the load. Walk
forward would have been out of the question.
Fortunately, I was able to change the money management aspect of the system
and move away from fixed number of positions. It's been trading profitably
ever since :)
Building on Herman's excellent suggestions, I would add that low end
machines (1.6 Ghz) can be had for under $150 these days (I got lucky on
eBay, paying $65 each for 7 of them!). Having a small farm allows you to
spread the load of a single walk forward (i.e. each machine handling a
different time period) or to backtest/optimize multiple different versions
of your strategy at the same time.
Mike
--- In amibroker@xxxxxxxxxxxxxxx, "Ken Close" <ken45140@xxx> wrote:
>
> A side question is this: what is the largest practical AFL file
ever created
> that runs and is used regularly?
>
> Obviously a hard question to answer (what is yours in terms of
number of
> lines or number of symbols operated on). We talk about memory and
multiple
> cores, and speed, etc, I will bet primarily in terms of
optimization speed.
> But what about just a large number of symbols and a large number of
> calculations and variables? That is what my question is aimed at.
>
> If you are wondering why this question, here is a brief
background: all of
> my recent postings have been because I am trying to work my way
towards
> implementation of what seems like a very large system. I want to
calculate,
> for each and every symbol, a ranking indicator or value really. I
want to
> use the "ordinal value" approach that I have recently been asking
about.
> Creating these ordinal values consumes a LOT of processing time and
> calculations etc. I would like to do this on 1000s of symbols
(maybe the
> entire database) but may have to settle for some subset, perhaps
only in the
> hundreds. But that is only the first step.
>
> After having these calculated values, I thought I would save them
into the
> OI field of each symbol. Then, separately, I would create an
elaborate
> rotational trading system, with a variety of parameters, and which
would use
> the ordinal ranking parameter I had previously calculated and saved
into the
> OI field. The rotational trading system, I am estimating, would
take a lot
> of statements and need to keep track of a lot of information, and
that would
> be before attempting any opimization of the trading parameters. I
am
> guessing that the rawbacktester mode is required.
>
> If separating the ranking calculation and saving it into the OI
field is a
> problem (because of the interplay between local storage vs plugin
access
> that I asked about in another message), then I could consider
putting
> everything (I mean everything) into one humongous AFL file.
Calculate
> ordinal rankings, keep them in their variables (by symbol) and then
go right
> into a rotational trading routine, all in the same code file. I
suspect it
> would either overload memory, or would take days of continual
running to
> complete.
>
> That is why I asked what is the largest code file you have ever
made. Maybe
> my quest is a futile one, but I am still plowing along.
>
> Thanks for listening.
>
> Ken
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|