HTCondor Project List Archives



[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-devel] sorting of groups in negotiation cycle





On 5/23/11 12:20 PM, Erik Erlandson wrote:
On Mon, 2011-05-23 at 11:59 -0500, Dan Bradley wrote:
Hi Erik,

It used to be the case that groups were sorted by the fraction of their
quota that they were using.  Those starving the most were considered
first.  I don't spot anywhere in the code where it is now sorting in
this way.  Am I overlooking something?  Or has this intentionally changed?
Hi Dan,

It wasn't intentional -- restoring some kind of sorting order fell
through the cracks during the overhaul to implement HGQ.   This is also
related to Brian Bockleman's requests in our other thread.

My preferred solution to this would be to re-factor the interaction of
HGQ with computation of submitter limits, so that negotiateWithGroup
loops over *all* submitters at once, across all groups, but the
computation of submitter limits is altered so that it always takes group
quotas into account.  This will allow all submitters to negotiate in
priority order (which I assume is what happened back in deep fossil time
prior to accounting groups), but group quota limits will still be
respected.

Interesting. It sounds like this needs some careful thinking. In some uses of the old flat group accounting setup, the user priorities were intentionally wildly different from the quotas. In other words, the surplus round in which everyone vied for the same pie was ordered quite differently from the ordering during the group negotiation phase. This is something I wish had been in bold letters in my head when we discussed early versions of the HGQ patch. I don't know if this same use-case exists in the new system, but if it does, it would be good to think about how it would interact with your proposed change.

However, if we have users who would like a faster solution, I think it
would be a quicker and safer fix to restore some sorting at the group
level.  Fraction of quota used seems like a win for preventing group
starvation, although I don't think it would address the CMS use case.

The safer fix sounds good for 7.6.  I've created a ticket for that:

https://condor-wiki.cs.wisc.edu/index.cgi/tktview?tn=2186

--Dan