Re: [HTCondor-users] Filling pool breadth-first with partitionable slots

[Jason Liu <jason.liu@xxxxxxxxxxxxx>]

Hi All,

Thanks for all your help.

I confirm 

NEGOTIATOR_POST_JOB_RANK = +MY.Cpus

indeed results in breadth-first-fill of our HTCondor pool, which is exactly what I was looking for.

There is just one caveat: HTCondor 8.4.7 has a default NEGOTIATOR_PRE_JOB_RANK of:

(10000000 * My.Rank) + (1000000 * (RemoteOwner =?= UNDEFINED)) - (100000 * Cpus) - Memory

If I am not mistaken this causes a depth-first-fill. So in my condor_config I have changed it:

NEGOTIATOR_PRE_JOB_RANK = 0

NEGOTIATOR_POST_JOB_RANK = +MY.Cpus

I can also confirm the NEGOTIATOR_PRE_JOB_RANK and NEGOTIATOR_POST_JOB_RANK work well with CONSUMPTION_POLICY and concurrency limits. I have these in my condor_config:

NUM_SLOTS = 1 

NUM_SLOTS_TYPE_1 = 1 

SLOT_TYPE_1 = cpus=100% 

SLOT_TYPE_1_PARTITIONABLE = true 

CONSUMPTION_POLICY = True

NetworkBandwidth_LIMIT = 1000

NEGOTIATOR_PRE_JOB_RANK = 0

NEGOTIATOR_POST_JOB_RANK = +MY.Cpus

And the overall effect is exactly what I want: breadth-first-fill based on CPU with pool-wide NetworkBandwith resource of 1000 units.

Thanks again for everyoneâs input.

Kind Regards

Jason

We are looking for a way to fill our pool breadth first, without the

limitation of only being able to match one job per negotiation cycle

per machine. Does HTCondor have a way to overcome this?

We are using HTCondor 8.4.7. In addition, we are also using

concurrency limits and consumption policy is enabled on all machines

in the pool.

Any help would be greatly appreciated.

Thanks.

Kind Regards, Jason

Hi Jason,

**Because your pool has enabled consumption policy**, I think Tom's 

suggestion below should enable you to setup depth-first filling and 

still match many jobs per negotiation cycle per machine.  Please let us 

know how it goes, and assuming it works for you (I think it will), I 

will update the HOWTO recipe you referenced on the wiki.

regards,

Todd

On 10/31/2016 10:02 AM, Tom Downes wrote:

On the negotiator, I'd do something like this:

NEGOTIATOR_POST_JOB_RANK = +MY.Cpus

This makes the most favorable match the partitionable slot with the most

CPUs which is what I mean by "breadth first." You can obviously extend

this idea to memory, but you need to scale the memory to CPUs. For me, I

have 2GB of RAM per core so this would be

NEGOTIATOR_POST_JOB_RANK = +MY.Cpus + (MY.Memory / 2048)

--

Tom Downes

Senior Scientist and Data Center Manager

Center for Gravitation, Cosmology and Astrophysics

University of Wisconsin-Milwaukee

414.229.2678

PRIVACY AND CONFIDENTIALITY NOTICE
The information contained in this message is intended for the named recipients only. It may contain confidential information and if you are not the intended recipient, you must not copy, distribute or take any action in reliance on it. If you have received this message in error please destroy it and reply to the sender immediately or contact us at the above telephone number.
VIRUS DISCLAIMER
While we take every precaution against presence of computer viruses on our system, we accept no responsibility for loss or damage arising from the transmission of viruses to e-mail recipients.