[HTCondor-users] Preempt a job when memory usage to higher than requested, only if total system memory is getting low

[Charles Goyard <cgoyard@xxxxxxx>]

Hi,

now that we have dynamic slots on our pool, we enjoy the noisy neighbor 
problem.

That is, some users correctly set their request_memory parameter, and 
some don't. This can lead to an unfair situation where badly configured 
jobs penalize the good citizens.

I found out the configuration template to evict jobs that use more than 
requested, and I'm planning to put is to good use. But let's add a grain 
of salt.

What I would like to achieve, is to allow jobs to eat more cake that 
expected, as long as there is no memory pressure at the system level (a 
bit like how group quota surplus work).

How can I come up with an expression that evaluates the total free (or 
used) memory on a compute node? Can I gather memory information from 
other slots?

Something like :

PREEMPT=( (MemoryUsage > Memory) && ( SumOfMemoryUsageAcrossSlots > ( 
TotalComputerMemory * 0.95 ) ) )


Thanks !

--
Charles