There are a lot of settings it seems, from PREEMPTION_REQUIREMENTS to MaxJobRetirementTime to PREEMPT_LATENCY. It is just not all that clear to me how to go about getting all these settings to do what I want as far as putting an upper limit on jobs after which they can not be guaranteed to run, while at the same time not kicking off jobs that may be running long for legitimate reasons on an otherwise underused cluster.
Terrence