Re: [HTCondor-users] Best Practice method to limit number of slots on a machine

["Beyer, Christoph" <christoph.beyer@xxxxxxx>]

Hi Lyle,

you can limit the number of CPUs that are used to create slots (that's if you use partitionable slots) using:

NUM_CPUS -> lie about the number of detected cpus

MAX_NUM_CPUS -> limit the max number of cpus

The numbers should be integer - hence I am not sure if something like '0,75 *  $(DETECTED_CPUS)' will work - you have to try it I guess ;)

If you have static slots it is very easy, just create less slots ...

Best

christoph

--
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx

---

Von: "Lyle Pakula" <Lyle@xxxxxxxxxxxxxxxx>
An: "HTCondor-Users Mail List" <htcondor-users@xxxxxxxxxxx>
Gesendet: Freitag, 9. September 2022 00:43:01
Betreff: [HTCondor-users] Best Practice method to limit number of slots on a        machine

Hi Everyone,

I'm wondering what people's best practice is to easily limit the number of jobs running on a machine as not to saturate that machine. 

Our jobs are heavy I/O rather than compute. So when a job is submitted we try to spread it across the cluster using the below rank

# Define Load Balancing on the AE Pool

NEGOTIATOR_PRE_JOB_RANK=( \$(NEGOTIATOR_PRE_JOB_RANK) ) * SlotID

But big jobs will still saturate. So what's the best way to limit the number of slots use on a machine, noting our machines vary from 4-8 cores. Ideally we would limit the number of slots ot say 75% of what the max is. Below is a bit heavy handed.

# DON'T COUNT HYPERTHREADED CPUS AS THIS LEADS TO SATURATION 
COUNT_HYPERTHREAD_CPUS=FALSE

Thanks, Lyle

--

AE CAPITAL
15 William St, Level 19, Melbourne VIC, Australia

p +61 3 9020 7801
m +61 (0)434 872 054

w http://www.aecapital.com.au

AE Capital Pty Limited (ACN 153 242 865) is regulated by the Australian Securities & Investments Commission and is a Corporate Authorised Representative of JFM Pty Limited (ACN 125 150 656), holder of an Australian Financial Services Licence (AFSL 314585).  AE Capital Pty Limited is a member of the National Futures Association (ID 0498660).

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/