[HTCondor-users] Question about VirtualMemory

[Di Qing <dqing@xxxxxxxxx>]

Hi All,

I am trying to find a way to limit the memory usage of the jobs and 
avoid they use too much swap space. I am testing HTCondor 8.4.11 with 
CGroup under SL6. The node has been partitioned to two static slots with 
different memory and CPU assignment. Each static slot can be further 
partitioned into smaller dynamic slots according to the CPU and memory 
requests of jobs.

In my test, with hard limit of CGroup, when configuring the slot 
partitionable, when job running, the memory.limit_in_bytes is set to the 
memory requested by the job and memory.memsw.limit_in_bytes is set to 
the total virtual memory of the node, i.e., physical memory plus swap 
space, although I configured its swap space to some percentages or set 
PROPORTIONAL_SWAP_ASSIGNMENT to true. If I configured the slot is not 
partitionable, the memory.limit_in_bytes of job is set to the assigned 
physical memory of
slot and memory.memsw.limit_in_bytes is set to the virtual memory of the 
slot.

With soft limit of CGroup,  both memory.limit_in_bytes and 
memory.memsw.limit_in_bytes set to the total virtual memory of the node 
(not the slot) when configuring the slot partitionable. Also the 
memory.soft_limit_in_bytes is set to the memory requested by job. When 
the slot is not partitionable, both memory.limit_in_bytes and 
memory.memsw.limit_in_bytes set to the virtual
memory assigned to the slot, and memory.soft_limit_in_bytes is set to the
physical memory of slots.

My question is why HTCondor always sets the  memory.memsw.limit_in_bytes 
to the total virtual memory of the node not the slot when the slot is 
configured partitionable? We don't want to completely disable the swap 
device since OS or some light tasks can still use some swap space.

Thanks,

Di