Re: [HTCondor-users] negotiator questions

["Krieger, Donald N." <kriegerd@xxxxxxxx>]

Thanks - very helpful.

Best - Don

> -----Original Message-----
> From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf
> Of Brian Bockelman
> Sent: Monday, March 07, 2016 9:55 AM
> To: HTCondor-Users Mail List
> Subject: Re: [HTCondor-users] negotiator questions
> 
> 
> > On Mar 6, 2016, at 8:45 AM, Krieger, Donald N. <kriegerd@xxxxxxxx> wrote:
> >
> > Itâs my understanding that at each pass, the negotiator goes through each
> queued job to decide whether and which job slot to assign it.
> > I use the directive, Priority = 0-n to control which jobs run first but I also have
> 2 groups of jobs with disjoint IP address requirements to divide my jobs
> between two subsets of the condor pool.
> > My question is about the order and âcompletenessâ with which the negotiator
> âexaminesâ my queued jobs.
> >
> > I presume that it goes through every single one of them regardless of how
> well and how many it has matched to job slots as it proceeds through.
> > Yes?
> 
> Mostly.  The exceptions are:
> 1) If there are other users in the system, you might run out of fair share during
> the negotiation cycle and the negotiator moves onto other usersâ jobs before it
> gets through your complete list.
> 2) The schedd the negotiator is talking to times out or has some other error.
> 
> >
> > I presume that it goes through them by âPriorityâ first and by fifo within each
> priority level.
> > Yes?
> 
> Mostly true.  You can assign other attributes (PreJobPrio* and PostJobPrio*)
> that affect the sort order for the jobs in the schedd.  Further, things get more
> complicated if you are submitting from multiple schedds.
> 
> >
> > I presume the time for each pass scales linearly as the total number of jobs
> queued.
> > Yes?
> >
> 
> Nope.  Identical jobs are grouped together into a single âresource requestâ.
> The negotiator builds a list of matches against the resource request, then sends
> them back to the schedd asynchronously.
> 
> Hence, the more âidentical jobsâ (typically, the jobs resulting from a âqueue Nâ
> statement in the submit file), the less the negotiation time.
> 
> > Can the negotiator get clobbered by a memory overrun due a very large
> number of queued jobs?
> 
> The negotiator, no.  It only considers a single resource request at a time - it
> does not buffer them all in memory (but it does buffer all machine ads in
> memory).
> 
> The schedd, however, is more sensitive to number of queued jobs.  Each takes
> up somewhere in the neighborhood of 50KB of RAM and a small amount of CPU
> time (which becomes noticeable once there are tens of thousands of idle jobs).
> 
> Brian
> 
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/