Re: [HTCondor-users] Limiting number of jobs of specific user to N per node

[John M Knoeller <johnkn@xxxxxxxxxxx>]

If there is a steady supply of jobs that fit in the slot after it has given up the extra resources (or concurrency limits), then what you would have is situation where you would starve the jobs requiring more resources.  This is very much like the problem we have now with dynamic slots where we need to occasionally drain them in order to allow jobs that need more resources to match.  In this case it would be only other jobs in the same schedd that would compete for the "smaller" slots. 

If you have ONLY one kind of job to run in a given schedd, you could try using the experimental slot pairing described here:
https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=ExperimentalOverlayedFileTransfer

The intended use case for these slots is to allow overlapping of transfer out with execution of the next job, but the slot pairing
mechanism is more flexible and could (in theory) be used for a pipeline of jobs that require more resources at the start of their life than they do at the end.  Please note that this is an experimental feature.  It has never been used in a production environment as far as I know.

-tj

-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Michael Pelletier
Sent: Tuesday, October 24, 2017 3:26 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] Limiting number of jobs of specific user to N per node

Ah, good point. I forgot about that aspect.

But if the slot had given up the resource, then wouldn't there be no match to the slot in the first place to give up?

How about being able to specify a resource or concurrency limit which expires at the end of the first (or Nth) job lease duration?

	-Michael Pelletier.

> -----Original Message-----
> From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf
> Of John M Knoeller
> Sent: Tuesday, October 24, 2017 11:24 AM
> To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
> Subject: [External] Re: [HTCondor-users] Limiting number of jobs of
> specific user to N per node
> 
> That's a feature request, and it wouldn't play well with slot re-use.
> If the slot gave up some resources once transfer input was finished, then
> the next job in the queue would not be able to use the slot for
> transferring input - it would have to give up the match and go back to the
> negotiator for a new one.
> 
> -tj
> 

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/