|
I made two runs each run consists of simultaneously
optimizing using AB (235 steps). The runs were started at the same time using
the window schedular and a _javascript_ with a built in timer.
1 run with 20 symbols and 235 steps core 1 =
57891millisec core2 = 57547 millisec
2nd run with 100 symobls and 235 steps core1 =
213813millsec core2 = 209141millisec
Difference is quite small < 3%
For those who are interested, here is the jscript that
runs it, just stick it with the schedular, it will calculate and display time
lapsed at the end of the run
var start = new Date();
var starttime = start.getTime();
var Ami = new ActiveXObject( "Broker.Application" );
AA = Ami.analysis;
var oldfilter1 = AA.Filter(0, "watchlist");
var oldfilter2 = AA.RangeMode;
AA.ClearFilters();
AA.ApplyTo = 2;
AA.LoadFormula("Z:\\Formulas\\Intest\\capitulation.afl");
AA.RangeMode = 3;
AA.Filter(0, "watchlist") = 12;
AA.optimize();
var finish = new Date();
var finishtime = finish.getTime();
var diff = finishtime - starttime;
WScript.Echo( "Timetaken = " + diff );
Some of you have
stated that you have run the same exhaustive search optimizations, I believe
using the same user account, simultaneously and have found that the
optimizations all finish at the same time.
My question is
…
Have any of you done
any testing to see if running the same exhaustive search optimization using
different user accounts finishes at the same time ? I’ve been playing
with this a little on two different machines and have noticed that on both
machines if I run the same optimization n times where n <= the number of
cores using a single user account that they will all finish at the same time.
However, for some reason when I use different user accounts all of which
have the same privileges etc i.e. the optimizations unexpectedly DON’T all
finish at the same time and in fact there is as much as a 20% difference in
run times. Have any of you noticed anything like this ? … If this isn’t
something you’ve played with and you have a few minutes in the not too distant
future I’d appreciate you giving it a shot and letting me know what you
find.
From:
amibroker@xxxxxxxxxps.com
[mailto:amibroker@xxxxxxxxxps.com] On Behalf Of Paul Ho
Sent: Saturday, May 24, 2008 11:56
PM
To:
amibroker@xxxxxxxxxps.com
Subject: RE: [amibroker] Re: Quad-core
test results
Fred
A couple
of questions
1. Can runas be used
on IO ie IO for use1 is connected to AB for user1 and IO for user2 is
connected to AB for user2?
2. regarding the
problem of multi-core CPU each running a copy of AB bearing down on a single
IO problem. You are today doing it with IOserver running on separate machine.
Cant you treat multi-core CPU as separate virtual machine. ie IOserver would
be runned on one core (actually as different user) accepting command from
IO from another core(user) through the TCP/IP
layer?
From:
amibroker@xxxxxxxxxps.com
[mailto:amibroker@yahoogroups.com] On Behalf Of Fred
Tonetti
Sent: Sunday,
25 May 2008 2:40 AM
To:
amibroker@xxxxxxxxxps.com
Subject: RE: [amibroker] Re: Quad-core
test results
Paul,
While this
method works, I would think it would be simpler to utilize something like
the RunAs command as this would allow all sessions to be viewable from a
single console w/o RDC or VNC etc i.e.
Assuming
there are userid’s IO1 through IOn defined
then
RunAs
/profile /user:IOn “c:\...\Broker.exe”
See the
attachment …
Beyond this
it doesn’t directly deal with the issue I really wanted to which is to bring
multiple cpu’s / cores from a single box to bear on a single problem using
the features of an intelligent Optimization. It does with AB alone
using an exhaustive search approach since this type of optimization problem
can be easily split into logical pieces and conceivably scripts could be
used to initiate the processes under each of the userid’s and gather the
results from them into a single place. I had assumed for the last
couple or three years that multi cpu / core utilization for optimization in
AmiBroker would be forth coming relatively soon but since that hasn’t
materialized maybe the best approach then is to use the tools that already
exist. Of course I’m sure if I spend the effort to do this that like
magic the functionality will suddenly appear where it more properly
belongs. Then again I suppose I should have just done this several
years ago.
From:
amibroker@xxxxxxxxxps.com
[mailto:amibroker@xxxxxxxxxps.com] On Behalf Of Paul Ho
Sent: Saturday, May 24, 2008 1:38
AM
To:
amibroker@xxxxxxxxxps.com
Subject: RE: [amibroker] Re: Quad-core
test results
Fred and
others
How to setup
multiple instances of IO/AB on Quad Core:
Since this might be
of interest to a few. I would post it here.
This is relatively
straight forward, the instructions below applies to XP and XP64. I dont use
Vista so I dont exactly know, but I expect it to be
similar
1. Create multiple
users accounts in XP, each user account will be used for each instance of
IO. You might decide to have a separate database for each user, but it is
not necessary. What is necessary is to setup IO on a different path for each
user to avoid overwriting the same files. I do that with the /IOpath
switch as an argument. For others who are interested in doing that refer to
Fred's doc. You can optionally open up task manager and set process affinity
to a particular core. I found that
unnecessary.
2. loggin in.
- There are 2 methods of logging in. Via console or remote desktop
connection. For the first user, just login in as usual, bring up AB and
start IO. Once AB/IO is running. You have move onto the 2nd user. Since XP
by default only allows 1 active user at a time. We have to "log out". If you
are at the console. It is important that you log out using switch user.
otherwise all your jobs will be killed. If you are using remote desktop, you
actually dont log out but disconnect by clicking the "x" button at the top.
Alternatively, if you're using XP32, you can download a patch on the
internet that allows multiple log ins, which means no disconnection is
necessary.
3. 2nd and
subsequent log in. Similar to the first.
4. Monitoring
multiple instances. if you have downloaded the patch and are able to run
multiple remote desktop connections. YOu can easily monitor the progress via
the connections that you have. Unfortunately,There is no patch for
XP64, So I am left with switching between users every now and then. But I
usually mount the IO directory as a network drive from another computer. so
I'm able to at least monitor two copies of
IOstatus.
From:
amibroker@xxxxxxxxxps.com
[mailto:amibroker@yahoogroups.com]
On Behalf Of Fred
Tonetti
Sent:
Friday, 23 May 2008 11:36 PM
To:
amibroker@xxxxxxxxxps.com
Subject: RE: [amibroker] Re:
Quad-core test results
Paul,
Interesting
and no I can’t say that I have looked into
it.
I would
think the drawback to this approach would be that while this very well
might work as you say, in an intelligent optimization the work being done
by each IO / AB / (CPU / Core) combination cannot really be cooperative
per se. This is because unlike an exhaustive search there is no way
to logically split an intelligent optimization except on a generation by
generation basis which implies there would have to be communication and/or
control between each generation i.e. in order for it to be cooperative I
would think there would need to be a higher level piece of software
controlling what was happening with each IO / AB combination … or … there
would need to be an IOServer like program running for each CPU / Core
beyond the first one that gets treated like a sort of local server to the
client.
From:
amibroker@xxxxxxxxxps.com
[mailto:amibroker@xxxxxxxxxps.com]
On Behalf Of Paul
Ho
Sent: Friday, May 23,
2008 2:37 AM
To:
amibroker@xxxxxxxxxps.com
Subject: [amibroker] Re: Quad-core
test results
--- In amibroker@xxxxxxxxxps.com,
Fred Tonetti <ftonetti@xx.>
wrote:
>
> PS .
>
>
>
> I have
noticed basically the same thing on dual core and core 2 duo
type
> machines i.e. the same optimization on a single symbol
can be run
> simultaneously on both cores with no increase in run
time .
>
>
>
> This seems to apply as well to
optimizations on hundreds of
symbols ..
>
>
I think
it all boils down to L2 Cache size, certainly the more
symbols + the
longer the data series, the larger L2 cache size has to
be. Certainly
I notice a marked increase in run time with both cores
running when
the machine is loaded up with symbols and/or with long
data series vs
few symbols and truncated data series.
>
> While the
tests that TJ performed seemed to imply otherwise, the
implication
> of the above would seem to be that it would be
beneficial for AB to
do the
> heavy lifting in terms of getting
multiple cpu's/cores involved in
an
> optimization when they
exist so that results could be combined in
one place.
>
>
I have been able to run multiple instances of IO, one on each cores
for about 2 years. All I need to do is to create a separate user
login per instance. I always thought that it was possible for IO to
treat multiple users on a single machine as multiple machines and use
TCP/IP services to communicate. Have you looked into this
possibility?
>
> If this were the case then IO would also
take advantage of this on
> individual machines as well as on
multiple machines simultaneously.
>
>
>
> _____
>
> From: amibroker@xxxxxxxxxps.com
[mailto:amibroker@xxxxxxxxxps.com]
On Behalf
> Of Steve Dugas
> Sent: Thursday, May 22, 2008
8:50 PM
> To: Yahoo - AmiBroker
> Subject: [amibroker]
Quad-core test results
>
>
>
> Hi All,
>
>
>
> I finished setting up the new quad-core machine
and ran my first
tests
> today, so am posting the promised test
results. All I can say is
Wow!
> Really nice improvements, even
better than I expected to see in my
highest
> hopes. I am happy
as a pig in sh*t !! 8 - )
>
>
>
> For
background, the test was an optimization run on a single
ticker,
which
> is how I will be running all my other tests ( ticker was
QID, 468
EOD data
> bars ). All tests used the same code with
different param settings,
all
> tests had about 46,000 opt
steps.
>
>
>
> To put things in perspective and
show why I am so happy, the
original code
> was about 2,400
lines. The first time I ran it, on my old backup
computer
>
which I was using as a dedicated optimization machine, it took 7
1/2
hours
> to run. So then I copied the code and created a shorter
version,
removing
> everything which wasn't absolutley necessary
for the optimizer, and
that
> reduced the run time to 2 1/2
hours. Then I ran this short version
on my
> faster primary
machine and that took 1 1/4 hours, which is about
what I was
>
expecting to see on the new machine.
>
>
>
> So
today I started by running just ran one instance on the quad-
core -
that
> took only 30 mins and Task Manager showed it was using just
25% of
the total
> processing power! Well to make a long story
short, I kept adding
one
> instance at a time, all instances ran
in 30 mins and each used up an
> additional 25% of the CPU power. In
the end, I was easily running 4
> simultaneous instances. This
pretty much kept the CPU tached at
100% but all
> instances ran
fine, all finished in 30 mins and I didn't experience
any
>
problems at all. I was even saving the first ones to spreadsheets
while the
> final ones were still finishing, wow everything just
worked
flawlessly! So,
> the quad-core can run 4 seperate opts
simultaneously in 30 mins,
which
> averages out to 7 1/2 minutes
per opt, which =
>
>
>
> 10X improvement over
running the short code on the fast machine...
>
> 20X
improvement over running the short code on the old dedicated
>
optimization machine...
>
> 60X improvement over running the
original code on the old
optimization
> machine... Awesome
!!!
>
>
>
> To those who were wondering what is
the best machine to get for
running AB,
> it looks to me like
quad is the way to go. ( My machine has an
Intel
> processor,
which TJ mentioned should probably work better than AMD
for
this
> stuff )....
>
>
>
> Steve
>
>
>
>
> _____
>
> I am using the
free version of SPAMfighter for private users.
> It has removed 459
spam emails to date.
> Paying users do not have this message in
their emails.
> Try SPAMfighter <http://www.spamfighter.com/len>
for free
now!
>
I am using the free version of SPAMfighter for
private users.
It has removed 459 spam emails to date.
Paying users
do not have this message in their emails.
Try SPAMfighter for free
now!
I am using the free version of
SPAMfighter for private users.
It has removed 460 spam emails to
date.
Paying users do not have this message in their emails.
Try SPAMfighter for free
now!
I am using the free version of
SPAMfighter for private users.
It has removed 460 spam emails to
date.
Paying users do not have this message in their emails.
Try SPAMfighter for free
now!
I am using the free version of SPAMfighter for
private users.
It has removed 460 spam emails to date.
Paying users do
not have this message in their emails.
Try SPAMfighter for free
now!
I am using the free version of SPAMfighter for private users. It has
removed 460 spam emails to date. Paying users do not have this message in
their emails. Try SPAMfighter
for free now!
__._,_.___
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
![]()
__,_._,___
|