[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [HTCondor-users] Prioritizing certain jobs on a subset of worker nodes
- Date: Wed, 05 Apr 2017 17:57:49 -0500
- From: Todd L Miller <tlmiller@xxxxxxxxxxx>
- Subject: Re: [HTCondor-users] Prioritizing certain jobs on a subset of worker nodes
Is there a Condor "best practice" to control the share of resources certain
jobs get of a subset of a pool's worker nodes?
Not as far as I know.
The only solution I can think of involves those worker nodes randomly
choosing a RANK expression to prefer certain jobs over others. I wonder
if there is something more elegant or centralized.
Since your worker nodes are glide-ins, you could configure them to
allow you to configure them remotely. If you also configured them so that
their initial START expression was FALSE (or advertised an initially-false
'Configured' attribute), you could then write a little script that polled
your collector looking for these glide-ins and build a site-by-site map of
job classes and counts. (You could add the job Class attribute to
STARTD_JOB_ATTRS so that it shows up in the machine ad to make this
easier.) At that point, you can do a little math and configure each
unconfigured startd to start only jobs of the class which would best meet
your policy.
Another option would be to run a central manager per site you care
about, and have your glide-ins report to the central manager corresponding
to the site they land on. Then you could use group quotas (where each
group is a job class) on each site-specific central manager to set your
policy, and have your schedd(s) flock to the central manager corresponding
to each site.
- ToddM