Re: [HTCondor-users] Limiting max number of running jobs for a group

[Antonio Dorta <adorta@xxxxxx>]

Hi Antonio,

some time ago we solved a similar situation using the Concurrency  
Limits  
(http://research.cs.wisc.edu/htcondor/manual/v8.6/3_14Setting_Up.html#48914)

Not sure if it is the best solution for your problem, but you can  
eventually give it a try, defining a limit in the negotiator and then  
setup a knob to add the proper 'concurrency_limits' to all jobs coming  
from the specific group.

Cheers,




Quoting Antonio Delgado Peris <antonio.delgadoperis@xxxxxxxxx>:

> Dear all,
>
> This is my first message to the list, so I'll start by presenting  
> myself :-)  I am writing from CIEMAT institute, at Madrid, Spain,  
> where we have recently installed a HTCondor cluster (with an  
> HTCondor-CE in front of it). We're still in the testing phase, but  
> should be moving to production fairly soon. We'll be serving mostly  
> (but not uniquely) the LHC CMS experiment.
>
> So moving to my question... we've defined some hierarchical dynamic  
> group quotas, with surplus allowed, which is nice because we want  
> minor groups to be able to use the farm if CMS is not running for  
> some reason. However, we also would like to limit their expansion,  
> so that they cannot occupy the whole farm (to speed up CMS taking  
> over the farm when their jobs come back).
>
> Naively, this would be like having both dynamic (soft, fair  
> share-like) quotas and static (hard) quotas for some groups. But the  
> manual says that if you define both dynamic and static quotas, the  
> dynamic one is ignored.
>
> I have looked for another parameter like  
> 'MAX_RUNNING_JOBS_PER_GROUP' but haven't found anything like that. I  
> have also tried to code some logic in the START expression using  
> 'SubmitterGroupResourcesInUse', but it didn't work (I think that  
> attribute is only usable for preemption... which we don't allow).
>
> We have solved the situation by just reserving some named nodes to  
> CMS, but I was still curious if there might be a less static  
> solution to the problem--i.e.: not tied to a fixed set of nodes, but  
> just stating a max number of simultaneous running jobs.
>
> Thanks for any hints. (And sorry if this question has been replied  
> earlier... I couldn't find it)
>
> Cheers,
>
>     Antonio
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/


--
Antonio Dorta
Servicios InformÃticos EspecÃficos (SIE)
InvestigaciÃn y EnseÃanza
Instituto de AstrofÃsica de Canarias (IAC)
C/ VÃa LÃctea, s/n. 38205 - La Laguna, Santa Cruz de Tenerife
Despacho: 1124. Tfno: 922 60 5278. email: adorta@xxxxxx
Supercomputing at IAC:  
http://www.iac.es/sieinvens/SINFIN/Main/supercomputing.php
----------------------------------------------------------------
ADVERTENCIA: Sobre la privacidad y cumplimiento de la Ley de  
Proteccion de Datos, acceda a http://www.iac.es/disclaimer.php
WARNING: For more information on privacy and fulfilment of the Law  
concerning the Protection of Data, consult  
http://www.iac.es/disclaimer.php?lang=en