Re: [HTCondor-users] All new MPI jobs become Idle when one existing MPI job Requirements are not met

[Greg Thain <gthain@xxxxxxxxxxx>]

On 06/08/2017 11:48 AM, Christophe DIARRA wrote:

My conclusion is that once there is an Idle MPI job, all the other submitted MPI job will remain also Idle even though the new submitted jobs requirements cant be met. The new Idle jobs are seen by condor_q -l as "not yet been considered by the matchmaker".

Is this the default behavior ? Is it possible to do something about it ? Any advice ?

This is the default behavior with the parallel universe. It intentionally schedules in strict FIFO order, to avoid starvation. Let's say the first job in the queue requires all the machines in the pool to run, and all the other jobs just require one machine. If the parallel scheduler skipped over the first job, the latter jobs would run, but the first job would potentially be starved forever and never run. If this is not a concern for you, this setting can be changed by setting

DEDICATED_SCHEDULER_USE_FIFO=false

in your condor_config file. With this setting, the dedicated scheduler will use a first fit algorithm, which may run more jobs, at the risk of potential starvation.

-greg