[HTCondor-users] Max Idle and DAGS

[Jeff Templon <templon@xxxxxxxxx>]

Hi,

Weâre trying to get out users to use the late materialisation factory stuff, to help avoid tens of thousands of queued jobs.

It doesnât seem to work though, with DAGs - even though the docs suggest it should.  Quoting one of our users:

I submitted these jobs using condor_submit_dag and setting the -MaxIdle flag as suggested when doing condor_submit_dag -h, as well as max_idle in the submission file of the sub-job. According to the documentation the flag is instead -maxidle so I also tried that to no avail.

According to the documentation these flags should set the DAGMAN_MAX_JOBS_IDLE key in the configuration. By looking at this key in the logs it appears that the flags are indeed ignored. Creating a local config file with DAGMAN_MAX_JOBS_IDLE = 20 changes the value in the logs, but from condor_q it still appears that more than 20 jobs can be idle. 

Any ideas?

Happy New Year,

JT