Carsten:
I too don't like hacky approaches to a specific problem. I would see how far something like this gets you:
1. Turn on IO accounting on the HTCondor cgroup (the parent one)
2. Create a ClassAd hook that monitors the stats coming out of that IO accounting
3. Figure out what "bad values" of yourÂstats are and impose some limits (in cgroups) that are below the bad value.
4. Create NEGOTIATOR_POST_JOB_RANK that puts machines approaching "bad" at the end of the list. You might have to combine width "breadth-first" filling policies based on CPUs to ensure that initial job matching goes that way.
5. A question for the crowd: since everything is an _expression_, could one modify the SLOT_WEIGHT so that I/O usage is included in the user's priority factor?
For 5,Âyou might scale the I/O weight so that "1000 IOPS = 0.5 CPUs" or anything below 1000 IOPS = 0, etc.
Look at IOAccounting, etc.. Read your actual OS man page to ensure you use settings appropriate to your release of SystemD.
Tom