Re: [HTCondor-users] portionable slots and greedy users

[Rita <rmorgan466@xxxxxxxxx>]

thankyou for your thoughts and responses.

On Mon, Oct 9, 2023 at 7:33âPM Todd L Miller via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:

> * mark a job as short . +JobType = "short". Is this the preferred way?

    The APs we run call it "JobDurationCategory", which is a wordier
but a little more specific. Depends on how much additional cleverness you
expect to add, I guess.

> * Short jobs have no quotas. They can run 100% of the pool if they like
> * If a job doesn't have +JobType, assume its "Long" type of job
> * Long jobs can only have 75% of the pool. Want to leave the rest of the
>Â Âpool empty for short users.
> * I want to prioritize jobs which have +JobType="short"

    If user A has only long jobs, and user B has only short jobs, but
user A has a better user priority, the negotiator _will_ give resources to
user before it gives them to user B. The only way I'm aware of to subvert
that even a little bit is by using accounting group quotas. Something
like the following might work:


# There are only two groups in the pool.
GROUP_NAMESÂ Â Â Â Â Â Â Â Â Â Â= group_long, group_short

# The 'short job' group can use at most 100% of the pool.
GROUP_QUOTA_DYNAMIC_group_short = 1.00

# The 'long job' group can use at most 75% of the pool.
GROUP_QUOTA_DYNAMIC_group_long = 0.75

# No, really, those percentages are hard limits.
# (This is a default value, but you do need it to be set this way.)
GROUP_ACCEPT_SURPLUSÂ Â Â Â Â Â = FALSE
# (This is a default value, but you do need it to be set this way.)
GROUP_AUTOREGROUPÂ Â Â Â Â Â Â Â= FALSE

# Always consider "group_short" first, then "group_long", then any
# jobs that slipped through without a group set.
GROUP_SORT_EXPRÂ Â Â Â Â Â Â Â Â= \
    ifThenElse( AccountingGroup=?="group_short", 1,
        ifThenElse( AccountingGroup=?="group_long", 10,
            100
        )
    )


You could pair this with a submit transform that converted JobType "short"
into accounting group "group_short" and everything else into "group_long",
or require that users set the accounting groups directly, maybe something
like:

use feature:AssignAccountingGroup(/path/to/mapfile)

where /path/to/mapfile countains:

* * group_long,group_short

which should assign all users' jobs into group_log by default, but allow
users to specify

accounting_group = group_short

in the submit files of short jobs.

-- ToddM
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

--

--- Get your facts first, then you can distort them as you please.--