You're not sorry - you enjoyed it ;-)
Exactly as I imagined it - a perfect example of a single AB task that
lends itself perfectly to, what I would call, a distributed approach.
I think that is about as simple as AB MCP could get since such
symmetry (repetition) of code would be rare, outside of opt?
Carefully avoiding any further reading of docs, I was also referring
to the fact that you must have utilized the OS to 'force' the ABnn
sessions (since we are in the accreditation mood I had better add I
think someone else kicked in the idea to use 'sessions' but what the
hell?).
You said that MCO was a pretty basic approach (or words to that
effect) but it seems to me that utilizing the capabilities of the OS
to distribute tasks to CPU resources, in various ways, can also be
done from within a program without the need for sessions.
As things stand at the moment couldn't you go ahead (if you wanted
to) and write an I/O plugin that utilizes all available cores WIWO
Tomasz changing anything in AB? (not to say it is a smart idea but
possible, yes?)
As a general comment - not particularly for you - what I am kicking
around is the idea that it would be a heck of a task to isolate the
bits of AB code that lend themselves to it, chop them up for
processing etc and then keep that up over the years (especially since
we won't all have the same OS/chip and/or number of cores) - I am
wondering of a less optimum but more achieveable path might be to use
the task scheduling capabilities of the OS to produce a more generic,
maintainable way to use all available cores.
The other point I am throwing on the table is - why would any
developer maintain backward compatibility for uniprocessors when as
of now no one is buyig one and realistically computers only have a
life of a few years.
brian_z
brian_z
FTR
I am not convinced that CMA-ES is the ants pants of optimization but
I thought I would sit on the sidelines and let you two guys slug it
out (of itself it might be nice and it could be a very good move as
part of an overall, yet to be seen AB strategy but I would say it has
some limitations) .
brian_z
--- In amibroker@xxxxxxxxxps.com,
Fred Tonetti <ftonetti@xx.>
wrote:
>
> Nope . Didn't do anything like that .
>
>
>
> What I did was simply as you state . where a sequence of code lends
itself
> to being chopped up, shoved through the processor and put back
together
> again on the other side .
>
>
>
> Even this description is not entirely accurate . What really
happens is that
> virtually the same AFL with different ranges of values for the
optimization
> statements is sent to multiple AB instances to be worked on by
different
> cores and then the results combined .
>
>
>
> From the doc ( Sorry to semi force you to read this )
>
>
>
> o MCO can deal with most forms of optimization statements.
MCO will
> check each of the Optimize statements in the AFL looking for the
one that
> has a numeric Min, Max and Increment arguments and needs the most
steps to
> get from the Min to Max value. Individual AFL's will be cloned for
each
> ABnn session with modifications to this Optimize statement so that
each core
> performs its share of the process. If for example the AFL to be
run had the
> following Optimize statements .
>
>
>
> Len1 = Optimize("Len1", 10, 1, 20, 1);
>
> Len2 = Optimize("Len2", 50, 1, 100, 1);
>
>
>
> . then it is the second Optimize statement that would be altered
for each of
> the AFL's to be used by the ABnn sessions. If one were using three
cores
> without any /load command line arguments then the resulting Optimize
> statement for each of the three AFL's would look like what is below
> effectively splitting the workload as closely as possible across
the user
> specified cores.
>
>
>
> Len2 = Optimize("Len2", 50, 1, 33, 1);
>
> Len2 = Optimize("Len2", 50, 34, 66, 1);
>
> Len2 = Optimize("Len2", 50, 67, 100, 1);
>
>
>
> _____
>
> From: amibroker@xxxxxxxxxps.com
[mailto:amibroker@xxxxxxxxxps.com]
On Behalf
> Of brian_z111
> Sent: Sunday, June 29, 2008 4:39 AM
> To: amibroker@xxxxxxxxxps.com
> Subject: [amibroker] Re: Another side to the multi-core support
argument
>
>
>
> Sorry, I used the term 'symmetric processing' incorrectly (I now
see
> that it has a formal definition whereas I used it informally).
>
> What I meant was that I don't think software applications for MCP
are
> limited only to single tasks where a sequence of code lends itself
to
> being chopped up, shoved through the processor and put back
together
> again on the other side.
>
> Couldn't software modularise its tasks/features and present them to
> the OS as if they are separate programs e.g. downloading RT data,
in
> AB, and saving to a local database, could be allocated one core, by
> the OS, and rendering that data into a chart/processing indicators
> allocated to another - possibly AB could use some smart
> prioritisation of its own tasks and work in tandem with the OS to
> prioritise access to core CPU's?
>
> It seems to me we are limiting ourselves with regard to the
> possibilities.
>
> (I haven't read, or tried, Freds MCO stuff - maybe that is what he
> did there already, in a basic way).
>
> If and when AB goes ahead with multicore application I have to
wonder
> why anyone wouldn't want to upgrade - could that be the time to end
> the unlimited backward compatibility rule (the benefits of
> maintaining backward compatibility for a few would no longer be
worth
> the effort and even detrimental to the many)?
>
> brian_z
>
> --- In amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
ps.com,
> "brian_z111" <brian_z111@> wrote:
> >
> > Re: what can traders do with multicore processing (MCP)?.
> >
> > Gee, where are all the Codesters when you need one?
> >
> > Are you guys holding out on us?
> >
> > MCP applications aren't limited to symmetric processing are they?
> >
> > http://en.wikipedia
<http://en.wikipedia.org/wiki/Computer_multitasking>
> .org/wiki/Computer_multitasking
> >
> > http://en.wikipedia <http://en.wikipedia.org/wiki/Multi-core>
> .org/wiki/Multi-core
> >
> > "The amount of performance gained by the use of a multicore
> processor
> > depends on the problem being solved and the algorithms used, as
> well
> > as their implementation in software (Amdahl's law). For so-
> > called "embarrassingly parallel" problems, a dual-core
processor
> with
> > two cores at 2GHz may perform very nearly as fast as a single
core
> of
> > 4GHz.[1] Other problems though may not yield so much speedup.
This
> > all assumes however that the software has been designed to take
> > advantage of available parallelism. If it hasn't, there will not
be
> > any speedup at all. However, the processor will multitask better
> > since it can run two programs at once, one on each core."
> >
> >
> > To what extent does it depend on the hardware/OS e.g. are the
> > possibilties for AB different on a MSFT system with an AMD chip
> > compared to another OS with a different chip - if so how would AB
> > handle that across a wide range of users - it seems to me the AB
> > solution will have to be generic rather than specialist?
> >
> >
> > As a layperson, it appears that multi-tasking is a more exciting
> > prospect, for the majority of traders, than optimising at warp
> speeds.
> >
> > Can't muliple processors handle multi-tasking at speeds unheard
of
> > before and hence bring some things into the realm of
possibilities?
> >
> >
> > How about:
> >
> > a) different databases running in one instance of AB - maybe even
> > with the databases in different timeframes/formats/coming from
> > different providers and being cross referenced in indicators.
> >
> > Can MCP allow users,who now run more than one installation of AB
on
> a
> > single computer (for whatever reason) to get the same results
from
> > one installation.
> >
> >
> > b) in the past Tomasz would enable AB's native database for extra
> > data (historical fields other than OHLCVOI) - as I understand it
> > because it would weigh AB down with things not used by the
majority.
> >
> > Perhaps MCP will make the weight of an open structure AB database
a
> > featherweight (PremiumData has already expressed an interest in
> > synching their database with AB so it is not as if there is no
> demand
> > for this feature).
> >
> > c) a task scheduler built into AB - set the tasks at the start of
> the
> > day - if the keyboard/processor is idle AB starts to run the
tasks
> > (backtesting/optimising).
> >
> > d) if we want to run our trading platform and AB on real time
with
> > two monitors on one computer do we have enough grunt to do that
now
> > OR would MCP allow us to run processing intensive tasks like that
> > concurrently OR maybe with a minimised AB chart (on tick data)
> > floating on top of our trading platform window.
> >
> > I don't know - I am merely speculating!
> >
> > Surely someone out there has some better ideas?
> >
> > brian_z
> >
> >
> >
> >
> >
> >
> >
> > --- In amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
ps.com,
> "sidhartha70" <sidhartha70@>
> > wrote:
> > >
> > > Hi Tomasz,
> > >
> > > I've been reading the arguments for and against multi-core
support
> > > with interest, and I just wanted to add my own voice to it and
> > mention
> > > an angle which hasn't currently been focused on.
> > >
> > > Since I bought AmiBroker a couple of months back I've been
blown
> > away
> > > by not only what a great piece of software it is, but also by
it's
> > > value and the fantastic customer support. Your own
responsiveness
> to
> > > customer needs and ability to efficiently add new features
leaves
> me
> > > with no hesitations in recommending AmiBroker to my fellow
> traders.
> > >
> > > However, I use AB in two ways. First for swing trading... for
> which
> > it
> > > is not only awesome, but also works perfectly happily on one
core
> > > because of the lower overhead in terms of data through put and
> > > therefore calculation overhead on that data.
> > >
> > > I also use AB for higher frequency day trading... and it's here
> for
> > me
> > > that multi-core support could really add something. For obvious
> > > reasons the multi-core argument has so far been almost
exclusively
> > > focused on optimization. However, for higher frequency traders
> like
> > > myself, working off just one core can be a significant
bottleneck.
> > >
> > > I have an IQ Feed 'tick' database... and some quite complex
chart
> > > setup's & indictaors. Currently, with chart refresh interval
set
> to
> > 2
> > > seconds (I'd like it quikcer) I can see one core of my computer
> > > working away at about 50%. However, if I increase the update
> > interval
> > > to zero to bring in every tick, I see useage on one core go up
to
> > > 80-90%. At this point, you can imagine, AB becomes noticeably
> clunky
> > > and difficult to use. Things get even worse if I start to think
> > about
> > > symbol linking. (for reference I am only looking at a couple of
e-
> > mini
> > > symbols)
> > > The frustrating aspect is that I have another 7 x 3Ghz cores
> sitting
> > > there twiddling their fingers while AB is grinding up on one
core.
> > >
> > > So, I do think, for those of us who use AB for higher frequncy
> > > trading, there is a very good argument to consider adding multi-
> core
> > > support outside of the optimization argument.
> > >
> > > Love to hear your (or anyone's!) thoughts on this.
> > >
> > > Many Thanks
> > >
> >
>
>
>
>
> _____
>
> I am using the free version of SPAMfighter for private users.
> It has removed 492 spam emails to date.
> Paying users do not have this message in their emails.
> Try SPAMfighter <http://www.spamfighter.com/len>
for free now!
>