[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] ENOENT writing to cgroup.subtree_control, but file exists



Hi Max,

being on 23.8.1. and with BASE_CGROUP set to a non-default path, we have the subtree virtual file [1] - might be due to the fix with 23.8.1 that Jamie mentioned? ð
(we have not set CREDD_HOST away from any default)

Cheers,
  Thomas

[root@batch1502 ~]# condor_config_val  BASE_CGROUP
system.slice/condor.slice
[root@batch1502 ~]# cat /sys/fs/cgroup/system.slice/condor.slice/cgroup.subtree_control
cpu io memory pids



On 10/07/2024 10.41, KÃhn, Max (SCC) wrote:
Hi all,

TLDR: We know whatâs triggering this and thereâs an obvious workaround. But weâre not sure whatâs the âproperâ way of running HTCondor with cgroupv2.

We did a bit of digging, the error from HTCondor is âcorrectâ just misleading. Even manually replicating the situation leads to the same error:

	$ echo "+cpu +io +memory +pids" > /sys/fs/cgroup/htcondor/cgroup.subtree_control
	-bash: echo: write error: No such file or directory

For whatever reason, our systems consistently have only âmemory pidsâ in /sys/fs/cgroup/cgroup.subtree_control. So thereâs a few pseudo-files missing for the cpu and io controllers, which kinda makes ENOENT appropriate.
Manually updating /sys/fs/cgroup/cgroup.subtree_control then makes HTCondorâs cgroup manipulation work. So a quick echo on boot is a workaround for now...

Does anyone know what is HTCondor's expected state for this? RH claims our state is the default on RHEL8 and RHEL9 [0,1]. Should the system, e.g. systemd, have enabled the controllers (RHEL8 ships with the dated systemd 239)? Should the BASE_CGROUP already be prepared with the appropriate controllers?

Cheers,
Max

[0] https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/managing_monitoring_and_updating_the_kernel/assembly_using-cgroupfs-to-manually-manage-cgroups_managing-monitoring-and-updating-the-kernel#proc_creating-cgroups-and-enabling-controllers-in-cgroups-v2-file-system_assembly_using-cgroupfs-to-manually-manage-cgroups

[1] https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/managing_monitoring_and_updating_the_kernel/using-cgroups-v2-to-control-distribution-of-cpu-time-for-applications_managing-monitoring-and-updating-the-kernel#using-cgroups-v2-to-control-distribution-of-cpu-time-for-applications_managing-monitoring-and-updating-the-kernel


On 8. Jul 2024, at 15:22, Matthias Schnepf <matthias.schnepf@xxxxxxx> wrote:

Hi Christoph,

It also happens when SE-Linux is disabled.

Cheers,

Matthias

On 7/8/24 13:39, Beyer, Christoph wrote:
Hi Max,

could this be related to SE-Linux ?

Best
christoph

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/


_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature