Re: [Condor-users] Evidence of Impact when running "nice" onwindows"

["Kewley, J \(John\)" <j.kewley@xxxxxxxx>]

Some jobs can proceed unnoticed, others can take over the machine.
Typically programs with large memory footprint (we have some scientific
codes that deliberately grab ALL available memory).

If you have multiproc machines such as the dual core models, you could also
try tuning condor so that jobs always run but only take over one of the 
processors and don't use the 2nd, that would help

Alternatively , have 1 processor act like default, but the other would alway run jobs.
I am not certain that would help with the memory hungry jobs like I
mentioned above though.

Cheers

JK

> -----Original Message-----
> From: condor-users-bounces@xxxxxxxxxxx
> [mailto:condor-users-bounces@xxxxxxxxxxx]On Behalf Of Bryan S. Maher
> Sent: Wednesday, September 27, 2006 3:27 PM
> To: Condor-Users Mail List
> Subject: Re: [Condor-users] Evidence of Impact when running "nice"
> onwindows"
> 
> 
> I'll add my comments even though they may contradict the previous
> responses. During August we had a major research push.  To meet the
> deadline I modified my Condor desktop policy to contribute all our
> desktop nodes fulltime to the queue.  
> 
> My personal desktop is a 2.26Ghz P4, with 1GB RAM, and running Windows
> XP SP2.  Our condor pool uniformly runs version 6.6.10.  When 
> jobs were
> running on my desktop it was nearly unusable.  Even for something as
> mundane as instant messenger, I was able to type dozens of words ahead
> of the computer which often missed keystrokes or even entire 
> words.  My
> Exchange server connection would constantly timeout eventually forcing
> me to work from my laptop just to read email.  Most of the jobs were
> Support Vector Model (SVM) training runs.  This is a fairly common
> statistical clustering technique which is memory intensive.  I believe
> this type of job impacts the system for two reasons:
> 
> 1) the large memory footprint forces other applications to swap out.
> 
> 2) because all operations are performed on in-memory data, 
> there are no
> wait states introduced for slow I/O devices which would free up the
> processor for other tasks.
> 
> I'm sure there are things we could have done to tweak the 
> condor policy
> to make our desktops a little more useful; however, the 10 days or so
> that we were inconvenienced did not justify the time it would 
> have taken
> to modify and test the policy -- especially given the fact we couldn't
> afford to have jobs fail during that time period due to policy
> misconfigurations.
> 
> Your mileage will vary.  Now that our major deadline has been met, I'm
> hoping to rollout 6.8.x to the pool with some policy 
> modifications which
> may address some of these issues.  It has been my experience 
> that Condor
> does require occasional tweaking to keep everyone happy.
> 
> Bryan Maher
> Carnegie Mellon University
> 
> P.S. This is not intended to be a negative commentary on 
> Condor.  We get
> a lot of work done using Condor and I recommend giving it a 
> try.  It may
> or may not be the tool for you but you won't know until you try it.
> 
> 
> -----Original Message-----
> From: condor-users-bounces@xxxxxxxxxxx
> [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Matt Hope
> Sent: Tuesday, September 26, 2006 3:49 PM
> To: Condor-Users Mail List
> Subject: Re: [Condor-users] Evidence of Impact when running "nice"
> onwindows"
> 
> On 9/26/06, James Osborne <osborneja1@xxxxxxxxxxxxx> wrote:
> > Hi All
> >
> > Does anybody out there have any evidence that documents the 
> impact, or
> > hopefully lack thereof of running Condor on Windows (or Linux) to
> always run
> > jobs - even when a user is logged in - but at the lowest 
> priority i.e.
> > job_renice_increment set to 19.  The reason I ask is that I 
> have been
> > conducting a small trial before hopefully going "run always" with a
> number of
> > our users and wondered if anbody had already done such a trial or
> would be
> > interested in the results of mine...
> 
> in addition to Ian's response
> 
> I often allow jobs to run on my machine overnight (dual Xeons, WinXP
> and 2GB memory) and others in my team have it set to run overnight.
> The jobs are very high CPU load (soak a single CPU fully for hours at
> a time) but are not terribly onerous on the network or disk. Most take
> around 400Mb, some can sometimes go as close to 2GB - all are .net
> applications.
> 
> I find I am unaware I have them running in the background unless I try
> to do something like a full build (where it tends to make it three
> times slower or so.
> 
> I have set it on all the other's machines to preempt the instant they
> touch the keyboard - On mine I simply don't let any more start. I find
> I kill off one vm maybe 1 day in 10.
> 
> If you have jobs with  large memory usage or heavy disk load I would
> expect that to be far more annoying to the users than CPU activity.
> 
> Previous experience with things like distributed.net leads me to say
> that windows is well capable of handling a low priority low memory
> footprint and disk activity app with the user never knowing (unless
> this makes their fan run louder :)
> 
> Matt
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to 
> condor-users-request@xxxxxxxxxxx with
> a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at either
> https://lists.cs.wisc.edu/archive/condor-users/
> http://www.opencondor.org/spaces/viewmailarchive.action?key=CONDOR
> 
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to 
> condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at either
> https://lists.cs.wisc.edu/archive/condor-users/
> http://www.opencondor.org/spaces/viewmailarchive.action?key=CONDOR
>