Re: [HTCondor-users] Improving negotiator match rate

[Greg Thain <gthain@xxxxxxxxxxx>]

On 1/19/21 6:32 AM, Zohar Kol wrote:

Thanks Todd,

After your inputs we understand that the matching rate is not what hurting us, but the job start rate.

But first let me describe our workload:

Our typical use-case would be every ~30 minutes burst submitting ~10K jobs.

Our 30 minutes job runtime histogram looks something like:

~0.8K more than 30 minutes

~3.5K 10-30 minutes

~2K 3-10 minutes

~2K less than 3 minutes

We are using the default CLAIM_WORKLIFE = 1200.

If the distribution of job sizes is relatively constant across the 30 minute bursts, you may want to consider using static slots, and configuring different sized static slots according to your job needs.  With those static slots, you could also consider the submission file setting keep_claim_idle to 30 minutes, which would keep the slot held by the schedd in the claimed/idle state, and ready for the next submission.  This assumes that all submissions are coming from one schedd and one user, and that your pool is dedicated to this work.

-greg