Re: [Condor-users] Evidence of Impact when running "nice" onwindows"

["Bryan S. Maher" <Bryan.Maher@xxxxxxxxxx>]

I'll add my comments even though they may contradict the previous
responses. During August we had a major research push.  To meet the
deadline I modified my Condor desktop policy to contribute all our
desktop nodes fulltime to the queue.  

My personal desktop is a 2.26Ghz P4, with 1GB RAM, and running Windows
XP SP2.  Our condor pool uniformly runs version 6.6.10.  When jobs were
running on my desktop it was nearly unusable.  Even for something as
mundane as instant messenger, I was able to type dozens of words ahead
of the computer which often missed keystrokes or even entire words.  My
Exchange server connection would constantly timeout eventually forcing
me to work from my laptop just to read email.  Most of the jobs were
Support Vector Model (SVM) training runs.  This is a fairly common
statistical clustering technique which is memory intensive.  I believe
this type of job impacts the system for two reasons:

1) the large memory footprint forces other applications to swap out.

2) because all operations are performed on in-memory data, there are no
wait states introduced for slow I/O devices which would free up the
processor for other tasks.

I'm sure there are things we could have done to tweak the condor policy
to make our desktops a little more useful; however, the 10 days or so
that we were inconvenienced did not justify the time it would have taken
to modify and test the policy -- especially given the fact we couldn't
afford to have jobs fail during that time period due to policy
misconfigurations.

Your mileage will vary.  Now that our major deadline has been met, I'm
hoping to rollout 6.8.x to the pool with some policy modifications which
may address some of these issues.  It has been my experience that Condor
does require occasional tweaking to keep everyone happy.

Bryan Maher
Carnegie Mellon University

P.S. This is not intended to be a negative commentary on Condor.  We get
a lot of work done using Condor and I recommend giving it a try.  It may
or may not be the tool for you but you won't know until you try it.


-----Original Message-----
From: condor-users-bounces@xxxxxxxxxxx
[mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Matt Hope
Sent: Tuesday, September 26, 2006 3:49 PM
To: Condor-Users Mail List
Subject: Re: [Condor-users] Evidence of Impact when running "nice"
onwindows"

On 9/26/06, James Osborne <osborneja1@xxxxxxxxxxxxx> wrote:
> Hi All
>
> Does anybody out there have any evidence that documents the impact, or
> hopefully lack thereof of running Condor on Windows (or Linux) to
always run
> jobs - even when a user is logged in - but at the lowest priority i.e.
> job_renice_increment set to 19.  The reason I ask is that I have been
> conducting a small trial before hopefully going "run always" with a
number of
> our users and wondered if anbody had already done such a trial or
would be
> interested in the results of mine...

in addition to Ian's response

I often allow jobs to run on my machine overnight (dual Xeons, WinXP
and 2GB memory) and others in my team have it set to run overnight.
The jobs are very high CPU load (soak a single CPU fully for hours at
a time) but are not terribly onerous on the network or disk. Most take
around 400Mb, some can sometimes go as close to 2GB - all are .net
applications.

I find I am unaware I have them running in the background unless I try
to do something like a full build (where it tends to make it three
times slower or so.

I have set it on all the other's machines to preempt the instant they
touch the keyboard - On mine I simply don't let any more start. I find
I kill off one vm maybe 1 day in 10.

If you have jobs with  large memory usage or heavy disk load I would
expect that to be far more annoying to the users than CPU activity.

Previous experience with things like distributed.net leads me to say
that windows is well capable of handling a low priority low memory
footprint and disk activity app with the user never knowing (unless
this makes their fan run louder :)

Matt
_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with
a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at either
https://lists.cs.wisc.edu/archive/condor-users/
http://www.opencondor.org/spaces/viewmailarchive.action?key=CONDOR