Re: [HTCondor-users] Limit jobs per node

[John M Knoeller <johnkn@xxxxxxxxxxx>]

As the person who added custom resources, I have to say I do not consider this a misuse.  

This trick requires coordination between job submission and the configuration of the execute nodes that may be hard for some pools to do in practice.  But for certain pools, this is an excellent solution to the problem of limiting certain jobs per node.

-tj

-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Beyer, Christoph
Sent: Wednesday, April 4, 2018 2:38 PM
To: htcondor-users <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] Limit jobs per node

Hi,

I guess machine_Resources could do the trick, on the workernode you would define: 

MACHINE_RESOURCE_NAMES = jobtype1, jobtype2, jobtype3
MACHINE_RESOURCE_jobtype1 = 1
MACHINE_RESOURCE_jobtype2 = 1
MACHINE_RESOURCE_jobtype3 = 1

In the jobtypes you have to request the limited resource: 

For jobtype1: 

request_jobtype1 = 1

And so on, hence, once a 'jobtype1' job is running on a machine the ressource jobtype1 is exhausted and no other job of this type will be able to run there (unless you set MACHINE_RESOURCE_jobtype1 to a higher value). Of cause it's a bit of a misusage and the feature is meant to serve 'real' limited ressources like licences or GPUs but who cares ;) 

Another maybe drawback is that you will never run more jobs of a certain type than there are machines in the pool but that is part of the idea I guess ...

Best
Chris

-- 
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx

----- UrsprÃngliche Mail -----
Von: "Heiko Schroeter" <heiko.schroeter@xxxxxxxx>
An: "htcondor-users" <htcondor-users@xxxxxxxxxxx>
Gesendet: Mittwoch, 4. April 2018 20:57:44
Betreff: Re: [HTCondor-users] Limit jobs per node

> 
> Another way to achieve this is to define a custom machine resource in the pool configuration, such as "OnePerHost" so that a job could do "request_oneperhost = 1" and be the only job running on the system, but that would apply to any job from anyone which requests "oneperhost" rather than only a given group of jobs.
> 
 >    -Michael Pelletier.

If i do understand this correctly then it means that i can run one job 
per node when requesting this special created Variable ? All other jobs 
are not limited which do not request it ?

That would be precisly our use case. Only one job per node for the user 
who is "requesting" it.
We do have lots of small jobs which do not have a large impact on the 
machines. But when it comes to large model simulations the user is happy 
if he can restrict his jobs in such a manner to only have one per node 
because of the I/O limitations.


Best
Heiko
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/