Re: [HTCondor-users] Limit jobs per node

[Michael Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx>]

If you make your request_disk expression take over half of the target machine's total disk space, then only one job will be able to fit on each system:

Request_disk = ifThenElse(isUndefined(TARGET.TotalDisk), 10485760, round(TARGET.TotalDisk * 0.51))

Hat-tip to Brian Bockelman for his "resizable jobs" HTCondor Week presentation.

So this expression will default to 1GB disk request when it's not being matched, and when being evaluated for a match it will request 51% of the total disk space of the remote host. A second match would request the same amount, and the target machine won't have enough left to satisfy that and thus move on to the next available host.

You can set that percentage to a higher value if you expect the job to use that much more disk space - 75%, etc.

You don't want to set it to 100% because that would prevent any other unrelated jobs which don't have high scratch space requirements from also running on the machine because each job requests a default amount of disk space for at least its Condor scratch directory contents, leaving capacity idle which might otherwise be usable.

Another way to achieve this is to define a custom machine resource in the pool configuration, such as "OnePerHost" so that a job could do "request_oneperhost = 1" and be the only job running on the system, but that would apply to any job from anyone which requests "oneperhost" rather than only a given group of jobs.

	-Michael Pelletier.

-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Mathieu Bahin
Sent: Wednesday, April 4, 2018 11:22 AM
To: htcondor-users@xxxxxxxxxxx
Subject: [External] Re: [HTCondor-users] Limit jobs per node

Thanks Thomas for the quick reply.
Though I'm not sure I made myself clear. My issue is not that 2 jobs are going to try to write the same file (with same filename I mean), I guess your solution address this problem (if I understood correctly...).
My problem is that if more than 1-2 jobs go to the same node, they will full this space, that's why I want the jobs to go on different nodes.

Cheers,
Mathieu

--
---------------------------------------------------------------------------------------
| Mathieu Bahin
| IE CNRS
|
| Institut de Biologie de l'Ecole Normale SupÃrieure (IBENS) Biocomp 
| team
| 46 rue d'Ulm
| 75230 PARIS CEDEX 05
| 01.44.32.23.56
---------------------------------------------------------------------------------------

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/