[HTCondor-users] More thoughts on memory limits

[Carles Acosta <cacosta@xxxxxx>]

Dear all,

We are running 23.10.1 version in all our EPs. We took the opportunityÂto add again a memory limit:

CGROUP_IGNORE_CACHE_MEMORY = True
MEMORY_EXCEEDED = (MemoryUsage isnt undefined && MemoryUsage > Memory*3)
use POLICY : WANT_HOLD_IF(MEMORY_EXCEEDED, 102, peak memory usage exceeded requested memory by 3 times)

The limit is generous, 3 times, becauseÂwe first want to test how this evolves.

After 3 weeks, it is clear that we do not have the huge overestimation of memory usage we saw in the past. However, it seems that the MEMORY_EXCEEDED _expression_ is generating some false positives. For instance, the same job wasÂsubmittedÂtwo times, the first time it shows a memory usage of 14 GB, and the second time, it shows a regular memory usage of 4 GB. I understand that this is the cgroups memory.peak, right? For CentOs7 or cgroupsv1, was the same max value considered (memory.max_usage_in_bytes) or the current (memory.usage_in_bytes)?Â

Does any other site use a limit like this? What is your experience?

Best regards,

Carles

--

Carles Acosta i Silva

PIC (Port d'Informacià CientÃfica)

Campus UAB, Edifici D

E-08193 Bellaterra, Barcelona

Tel: +34 93 581 33 08

Fax: +34 93 581 41 10

http://www.pic.esÂ

AvÃs - Aviso - Legal Notice: Âhttp://legal.ifae.es