|
PureBytes Links
Trading Reference Links
|
> I'm not sure what you mean by whether or not it might be a smart
>thing to do
I just meant that I am not absolutely sure about my bright ideas
since I don't have enough computing knowledge to be certain that I am
not making ridiculous statements.
Thanks for your answers - appreciated.
brian_z
--- In amibroker@xxxxxxxxxxxxxxx, Fred Tonetti <ftonetti@xxx> wrote:
>
> Ohhh it certainly wasn't my idea to use sessions . It was Paul's
offhand
> comment about already doing it that brought light to the subject
for me .
>
>
>
> If I had been aware enough to realize this could have been done I
would have
> done it years ago .
>
>
>
> In regards to having multithreading within individual programs .
yes of
> course . CAD/CAM programs have been doing this for years .
>
>
>
> In regards to doing the same thing in IO . Yes of course . MCO was
the proof
> of concept if you will . The capability within IO already exists
now in test
> mode . I'm not sure what you mean by whether or not it might be a
smart
> thing to do but imho it certainly is useful as is MCO to speed up
exhaustive
> search.
>
>
>
> While you may not have the same chip . you will have the same
underlying
> instruction set .
>
>
>
> As far as backward compatibility is concerned . I don't think this
is an
> issue . It is relatively simple to detect how many processors are
available
> and make use of more than one if it exists.
>
>
>
> I can't say that I see anything all that wonderful out of CMA-ES
either but
> that is of course just one person's opinion based on limited
testing. The
> problem with intelligent algorithms in general is that by and large
most are
> not all that wonderful in their originally published version for
the kinds
> of tasks that they'd be put to in AB . Some have very limited scope
and
> others have characteristics and limitations that make them of
limited
> usefulness as published . Thus many need to be played with to see
what makes
> them better and that playing takes oodles of time just to see if
some
> particular one is worthwhile pursuing .
>
> _____
>
> From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx]
On Behalf
> Of brian_z111
> Sent: Sunday, June 29, 2008 10:44 AM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: [amibroker] Re: Another side to the multi-core support
argument
>
>
>
> You're not sorry - you enjoyed it ;-)
>
> Exactly as I imagined it - a perfect example of a single AB task
that
> lends itself perfectly to, what I would call, a distributed
approach.
>
> I think that is about as simple as AB MCP could get since such
> symmetry (repetition) of code would be rare, outside of opt?
>
> Carefully avoiding any further reading of docs, I was also
referring
> to the fact that you must have utilized the OS to 'force' the ABnn
> sessions (since we are in the accreditation mood I had better add I
> think someone else kicked in the idea to use 'sessions' but what
the
> hell?).
>
> You said that MCO was a pretty basic approach (or words to that
> effect) but it seems to me that utilizing the capabilities of the
OS
> to distribute tasks to CPU resources, in various ways, can also be
> done from within a program without the need for sessions.
>
> As things stand at the moment couldn't you go ahead (if you wanted
> to) and write an I/O plugin that utilizes all available cores WIWO
> Tomasz changing anything in AB? (not to say it is a smart idea but
> possible, yes?)
>
> As a general comment - not particularly for you - what I am kicking
> around is the idea that it would be a heck of a task to isolate the
> bits of AB code that lend themselves to it, chop them up for
> processing etc and then keep that up over the years (especially
since
> we won't all have the same OS/chip and/or number of cores) - I am
> wondering of a less optimum but more achieveable path might be to
use
> the task scheduling capabilities of the OS to produce a more
generic,
> maintainable way to use all available cores.
>
> The other point I am throwing on the table is - why would any
> developer maintain backward compatibility for uniprocessors when as
> of now no one is buyig one and realistically computers only have a
> life of a few years.
>
> brian_z
>
> brian_z
>
> FTR
>
> I am not convinced that CMA-ES is the ants pants of optimization
but
> I thought I would sit on the sidelines and let you two guys slug it
> out (of itself it might be nice and it could be a very good move as
> part of an overall, yet to be seen AB strategy but I would say it
has
> some limitations) .
>
> brian_z
>
> --- In amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
ps.com, Fred
> Tonetti <ftonetti@> wrote:
> >
> > Nope . Didn't do anything like that .
> >
> >
> >
> > What I did was simply as you state . where a sequence of code
lends
> itself
> > to being chopped up, shoved through the processor and put back
> together
> > again on the other side .
> >
> >
> >
> > Even this description is not entirely accurate . What really
> happens is that
> > virtually the same AFL with different ranges of values for the
> optimization
> > statements is sent to multiple AB instances to be worked on by
> different
> > cores and then the results combined .
> >
> >
> >
> > From the doc ( Sorry to semi force you to read this )
> >
> >
> >
> > o MCO can deal with most forms of optimization statements.
> MCO will
> > check each of the Optimize statements in the AFL looking for the
> one that
> > has a numeric Min, Max and Increment arguments and needs the most
> steps to
> > get from the Min to Max value. Individual AFL's will be cloned
for
> each
> > ABnn session with modifications to this Optimize statement so
that
> each core
> > performs its share of the process. If for example the AFL to be
> run had the
> > following Optimize statements .
> >
> >
> >
> > Len1 = Optimize("Len1", 10, 1, 20, 1);
> >
> > Len2 = Optimize("Len2", 50, 1, 100, 1);
> >
> >
> >
> > . then it is the second Optimize statement that would be altered
> for each of
> > the AFL's to be used by the ABnn sessions. If one were using
three
> cores
> > without any /load command line arguments then the resulting
Optimize
> > statement for each of the three AFL's would look like what is
below
> > effectively splitting the workload as closely as possible across
> the user
> > specified cores.
> >
> >
> >
> > Len2 = Optimize("Len2", 50, 1, 33, 1);
> >
> > Len2 = Optimize("Len2", 50, 34, 66, 1);
> >
> > Len2 = Optimize("Len2", 50, 67, 100, 1);
> >
> >
> >
> > _____
> >
> > From: amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
ps.com
> [mailto:amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
ps.com]
> On Behalf
> > Of brian_z111
> > Sent: Sunday, June 29, 2008 4:39 AM
> > To: amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
ps.com
> > Subject: [amibroker] Re: Another side to the multi-core support
> argument
> >
> >
> >
> > Sorry, I used the term 'symmetric processing' incorrectly (I now
> see
> > that it has a formal definition whereas I used it informally).
> >
> > What I meant was that I don't think software applications for MCP
> are
> > limited only to single tasks where a sequence of code lends
itself
> to
> > being chopped up, shoved through the processor and put back
> together
> > again on the other side.
> >
> > Couldn't software modularise its tasks/features and present them
to
> > the OS as if they are separate programs e.g. downloading RT data,
> in
> > AB, and saving to a local database, could be allocated one core,
by
> > the OS, and rendering that data into a chart/processing
indicators
> > allocated to another - possibly AB could use some smart
> > prioritisation of its own tasks and work in tandem with the OS to
> > prioritise access to core CPU's?
> >
> > It seems to me we are limiting ourselves with regard to the
> > possibilities.
> >
> > (I haven't read, or tried, Freds MCO stuff - maybe that is what
he
> > did there already, in a basic way).
> >
> > If and when AB goes ahead with multicore application I have to
> wonder
> > why anyone wouldn't want to upgrade - could that be the time to
end
> > the unlimited backward compatibility rule (the benefits of
> > maintaining backward compatibility for a few would no longer be
> worth
> > the effort and even detrimental to the many)?
> >
> > brian_z
> >
> > --- In amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
> ps.com,
> > "brian_z111" <brian_z111@> wrote:
> > >
> > > Re: what can traders do with multicore processing (MCP)?.
> > >
> > > Gee, where are all the Codesters when you need one?
> > >
> > > Are you guys holding out on us?
> > >
> > > MCP applications aren't limited to symmetric processing are
they?
> > >
> > > http://en.wikipedia
> <http://en.wikipedia
<http://en.wikipedia.org/wiki/Computer_multitasking>
> .org/wiki/Computer_multitasking>
> > .org/wiki/Computer_multitasking
> > >
> > > http://en.wikipedia <http://en.wikipedia
> <http://en.wikipedia.org/wiki/Multi-core> .org/wiki/Multi-core>
> > .org/wiki/Multi-core
> > >
> > > "The amount of performance gained by the use of a multicore
> > processor
> > > depends on the problem being solved and the algorithms used, as
> > well
> > > as their implementation in software (Amdahl's law). For so-
> > > called "embarrassingly parallel" problems, a dual-core
processor
> > with
> > > two cores at 2GHz may perform very nearly as fast as a single
> core
> > of
> > > 4GHz.[1] Other problems though may not yield so much speedup.
> This
> > > all assumes however that the software has been designed to take
> > > advantage of available parallelism. If it hasn't, there will
not
> be
> > > any speedup at all. However, the processor will multitask
better
> > > since it can run two programs at once, one on each core."
> > >
> > >
> > > To what extent does it depend on the hardware/OS e.g. are the
> > > possibilties for AB different on a MSFT system with an AMD chip
> > > compared to another OS with a different chip - if so how would
AB
> > > handle that across a wide range of users - it seems to me the
AB
> > > solution will have to be generic rather than specialist?
> > >
> > >
> > > As a layperson, it appears that multi-tasking is a more
exciting
> > > prospect, for the majority of traders, than optimising at warp
> > speeds.
> > >
> > > Can't muliple processors handle multi-tasking at speeds unheard
> of
> > > before and hence bring some things into the realm of
> possibilities?
> > >
> > >
> > > How about:
> > >
> > > a) different databases running in one instance of AB - maybe
even
> > > with the databases in different timeframes/formats/coming from
> > > different providers and being cross referenced in indicators.
> > >
> > > Can MCP allow users,who now run more than one installation of
AB
> on
> > a
> > > single computer (for whatever reason) to get the same results
> from
> > > one installation.
> > >
> > >
> > > b) in the past Tomasz would enable AB's native database for
extra
> > > data (historical fields other than OHLCVOI) - as I understand
it
> > > because it would weigh AB down with things not used by the
> majority.
> > >
> > > Perhaps MCP will make the weight of an open structure AB
database
> a
> > > featherweight (PremiumData has already expressed an interest in
> > > synching their database with AB so it is not as if there is no
> > demand
> > > for this feature).
> > >
> > > c) a task scheduler built into AB - set the tasks at the start
of
> > the
> > > day - if the keyboard/processor is idle AB starts to run the
> tasks
> > > (backtesting/optimising).
> > >
> > > d) if we want to run our trading platform and AB on real time
> with
> > > two monitors on one computer do we have enough grunt to do that
> now
> > > OR would MCP allow us to run processing intensive tasks like
that
> > > concurrently OR maybe with a minimised AB chart (on tick data)
> > > floating on top of our trading platform window.
> > >
> > > I don't know - I am merely speculating!
> > >
> > > Surely someone out there has some better ideas?
> > >
> > > brian_z
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > --- In amibroker@xxxxxxxxx <mailto:amibroker%40yahoogroups.com>
> ps.com,
> > "sidhartha70" <sidhartha70@>
> > > wrote:
> > > >
> > > > Hi Tomasz,
> > > >
> > > > I've been reading the arguments for and against multi-core
> support
> > > > with interest, and I just wanted to add my own voice to it
and
> > > mention
> > > > an angle which hasn't currently been focused on.
> > > >
> > > > Since I bought AmiBroker a couple of months back I've been
> blown
> > > away
> > > > by not only what a great piece of software it is, but also by
> it's
> > > > value and the fantastic customer support. Your own
> responsiveness
> > to
> > > > customer needs and ability to efficiently add new features
> leaves
> > me
> > > > with no hesitations in recommending AmiBroker to my fellow
> > traders.
> > > >
> > > > However, I use AB in two ways. First for swing trading... for
> > which
> > > it
> > > > is not only awesome, but also works perfectly happily on one
> core
> > > > because of the lower overhead in terms of data through put and
> > > > therefore calculation overhead on that data.
> > > >
> > > > I also use AB for higher frequency day trading... and it's
here
> > for
> > > me
> > > > that multi-core support could really add something. For
obvious
> > > > reasons the multi-core argument has so far been almost
> exclusively
> > > > focused on optimization. However, for higher frequency
traders
> > like
> > > > myself, working off just one core can be a significant
> bottleneck.
> > > >
> > > > I have an IQ Feed 'tick' database... and some quite complex
> chart
> > > > setup's & indictaors. Currently, with chart refresh interval
> set
> > to
> > > 2
> > > > seconds (I'd like it quikcer) I can see one core of my
computer
> > > > working away at about 50%. However, if I increase the update
> > > interval
> > > > to zero to bring in every tick, I see useage on one core go
up
> to
> > > > 80-90%. At this point, you can imagine, AB becomes noticeably
> > clunky
> > > > and difficult to use. Things get even worse if I start to
think
> > > about
> > > > symbol linking. (for reference I am only looking at a couple
of
> e-
> > > mini
> > > > symbols)
> > > > The frustrating aspect is that I have another 7 x 3Ghz cores
> > sitting
> > > > there twiddling their fingers while AB is grinding up on one
> core.
> > > >
> > > > So, I do think, for those of us who use AB for higher frequncy
> > > > trading, there is a very good argument to consider adding
multi-
> > core
> > > > support outside of the optimization argument.
> > > >
> > > > Love to hear your (or anyone's!) thoughts on this.
> > > >
> > > > Many Thanks
> > > >
> > >
> >
> >
> >
> >
> > _____
> >
> > I am using the free version of SPAMfighter for private users.
> > It has removed 492 spam emails to date.
> > Paying users do not have this message in their emails.
> > Try SPAMfighter <http://www.spamfigh
<http://www.spamfighter.com/len>
> ter.com/len> for free now!
> >
>
>
>
>
> _____
>
> I am using the free version of SPAMfighter for private users.
> It has removed 492 spam emails to date.
> Paying users do not have this message in their emails.
> Try SPAMfighter <http://www.spamfighter.com/len> for free now!
>
------------------------------------
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|