Re: [HTCondor-users] How to modify cpus

[Tom Downes <downes@xxxxxxx>]

--

Tom Downes
Senior Scientist and Data CenterÂManager
Center for Gravitation, Cosmology and Astrophysics
University of Wisconsin-Milwaukee
414.229.2678

On Sun, May 15, 2016 at 1:23 PM, Brian Bockelman <bbockelm@xxxxxxxxxxx> wrote:

> That's unless your jobs are doing significant amounts of other activity (in particular disk or network I/O), in which case you may need to pretend that there are more cores than you really have in order to utilise them fully.

Yup, all sorts of things can depend on the workload - and can go awry when the workload changes (spoken as a person who overcommits memory).

I overallocate memory by lying to Condor that it has 64 GiB (powers of 2) of RAM available to it when, in fact, it has something between 64000 MiB and 65000 MiB owing to the OS itself and some having been stolen by the BIOS/video card. I make up for this fact, by using cgroups to restrict Condor jobs to a total of 64000 MiB of RSS by enabling memory.use_hierarchy.

Best compromise I could find between job matching efficiency -- I now round requests off into powers of 2 -- and avoiding chaos.

Tom