Re: [HTCondor-users] Flock ing concurrency

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 3/24/2023 2:34 AM, Peter Ellevseth wrote:

Hi all

 

We are setting a supplementary server for times when our original cluster is not enough. Intention was to use flocking for this. We are running licensed software on the cluster and employing concurrency limits to ensure that condor doesnât start more jobs than we have licenses for. As I understand it, concurrency is per pool, so the new pool we want to flock to will no reflect the concurrency limits in the original pool.

    Hi Peter,
    
    You are correct, concurrency limits are per pool (specifically, per
    condor_negotiator).  
    
    I guess my thought is not bother with flocking, and achieve your
    goal of only utilizing some additional supplemental servers via an
    alternative configuration.
    
    As an alternative, I am thinking just put all the servers into one
    pool.  
    First in the config file on your central manager, tell your central
    manager to allocate execute nodes "depth first", i.e. try to full up
    all the slots on a server before moving to another.
    To do this, follow the recipe at
https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=HowToFillPoolDepthFirst
    
    Next, in the HTCondor config on your supplementary servers (i.e.
    your supplementary execute nodes), add the following:
    
        # Set a negative value for the condor_startd RANK, so that the
    central manager will avoid matching slots
        # on this supplemental machine unless everything else is busy...
        RANK = -500 
    
    Hope the above helps,
    Todd