Re: [HTCondor-users] Can condor still create cgroups when run inside a docker container?

[Boris Sadkhin <bsadkhin.anl@xxxxxxxxx>]

Greg,

Thanks for the hint, because of that I was able to figure out what were the causes of the issue.

 

1) I was using the andypohl htcondor docker image, (based on centos) which had a configuration value that disabled cgroups with BASE_CGROUP="" (So I had to make sure BASE_CGROUP="" was not set to an empty string.)

2) I was also running the condor daemons as a non root user, so I had to run the condor_master using root for the execute containers, so it could start up the startd daemons as root..

3) Lastly, I had to volume mount /sys/fs/cgroup:/sys/fs/cgroup for the docker container containing the startd daemons

 

So I now have been able to successfully run HTCondor (in a docker container with a CentOS base) on an CentOS host with cgroups being set automatically for each slot.

However, when running HTCondor in a docker container on an Ubuntu host, I am having issues with cgroups again, namely the memory.limit_in_bytes not being set on an ubuntu host.

 

So any hints about that would be appreciated. Hereâs what I found out when running a centos docker container with HTCondor on a centos vs an ubuntu host:

 

On my 8GB CENTOS virtual machine with 3 slots, the cgroups are set as follows, which seems OK to me.

 

=========================================
WITH CGROUP_MEMORY_LIMIT_POLICY = unset
memory.limit_in_bytes = 9223372036854771712

memory.soft_limit_in_bytes = 9223372036854771712

 

WITH CGROUP_MEMORY_LIMIT_POLICY = soft

memory.limit_in_bytes = 4293959680 *4G

memory.soft_limit_in_bytes = 2839543808 *2.7G

WITH CGROUP_MEMORY_LIMIT_POLICY = hard

memory.limit_in_bytes = 2839543808 *2.7G

memory.soft_limit_in_bytes = 0

=========================================

 

But on my Ubuntu Virtual Machine, I have condor running in a container and the cgroups are as follows

 

=========================================

WITH CGROUP_MEMORY_LIMIT_POLICY = unset
memory.limit_in_bytes = 9223372036854771712

memory.soft_limit_in_bytes = 9223372036854771712

WITH CGROUP_MEMORY_LIMIT_POLICY = soft
memory.limit_in_bytes = 9223372036854771712

memory.soft_limit_in_bytes = 2839543808 *2.7G

 

WITH CGROUP_MEMORY_LIMIT_POLICY = hard

memory.limit_in_bytes = 9223372036854771712

memory.soft_limit_in_bytes = 0

=========================================

 

As you can see, only the memory.soft_limit_in_bytes is being set by HTCondor.  Any clues on what to check out next would be appreciated.

 

Thank you for your help.