Re: [HTCondor-users] Future improvements to NUMA affinity algorithms

[Valerio Bellizzomi <valerio@xxxxxxxxxx>]

On Wed, 2025-07-30 at 18:20 -0500, Greg Thain via HTCondor-users wrote:
> 
> On 7/30/25 4:51 PM, Jaime Frey via HTCondor-users wrote:
> > My suspicion is that this wonât have much effect on the difference
> > in CPU scheduling efficiency between ASSIGN_CPU_AFFINITY and
> > cgroups for enforcing CPU usage limitations.
> > HTCondor will still ignore NUMA locality when selecting which cores
> > to assign to a job.
> > And ASSIGN_CPU_AFFINITY doesnât allow a job to use additional cores
> > when theyâre idle (if job has more processes/threads than assigned
> > cores).
> 
> 
> My hope is that algorithms like this will allow HTCondor and really
> any 
> job manager to work better on NUMA machines without the admin having
> to 
> set ASSIGN_CPU_AFFINITY. The kernel is always going to have a better
> idea about the memory topology of any given system than HTCondor
> does. 
> So, in a perfect world, we leave core-scheduling to the kernel, and
> my 
> reading of this change is that it helps the kernel schedule a bit
> better 
> without devolving to forcing processor affinity to a job.
> 
> 
> -greg

When the kernel 6.17 will be available I will make an experiment on my
little cluster, I will try setting ASSIGN_CPU_AFFINITY and without
setting to see what the difference is.

Ideally it would be useful to have some method to monitor which cores
are being assigned to a job at runtime.

Kind Regards
Valerio



> 
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to
> htcondor-users-request@xxxxxxxxxxxÂwith a
> subject: Unsubscribe
> 
> The archives can be found at:
> https://www-auth.cs.wisc.edu/lists/htcondor-users/