Re: [HTCondor-users] MAX_JOBS_SUBMITTED exceeded, submit failed. Current total is 499999. Limit is 50000

["Beyer, Christoph" <christoph.beyer@xxxxxxx>]

Hi,

it seems to me - at least on my scheds the MAX_JOBS_SUBMITTED is indeed the number of jobs the sched dealt with since the last boot (I suppose)

At least this is definetley not the current number of jobs on this sched:

[root@bird-htc-sched11 ~]# condor_config_val MAX_JOBS_SUBMITTED
2147483647

;)

Hence it looks to me as if MAX_JOBS_SUBMITTED should not be set at all unless you want to stop the scheduling after a certain amount of jobs ?

Maybe MAX_JOBS_PER_OWNER is more likely to do what you want (limiting the number of jobs per owner on the sched) ?

Best

christoph

--
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx

---

Von: "Vikrant Aggarwal" <ervikrant06@xxxxxxxxx>
An: "HTCondor-Users Mail List" <htcondor-users@xxxxxxxxxxx>
Gesendet: Mittwoch, 8. MÃrz 2023 08:06:16
Betreff: Re: [HTCondor-users] MAX_JOBS_SUBMITTED exceeded, submit failed. Current total is 499999. Limit is 50000

Thanks Jamie,

But we don't have this many jobs in the queue. The batch we are trying to submit has only a handful of jobs still we are hitting the max job limit. 

03/07/23 21:46:46 (pid:55697) NewCluster(): MAX_JOBS_SUBMITTED exceeded, submit failed. Current total is 300027. Limit is 300000

03/07/23 22:11:09 (pid:55697) NewCluster(): MAX_JOBS_SUBMITTED exceeded, submit failed. Current total is 300000. Limit is 300000

It's happening randomly but often on a few submit nodes (not all). All submit nodes are with the same conf.  

Thanks & Regards,

Vikrant Aggarwal

On Wed, Feb 8, 2023 at 9:24âPM Jaime Frey via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:

> On Feb 7, 2023, at 11:01 AM, Todd L Miller via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:
>
>> We hit this issue multiple times: Issue disappears if we restart the condor
>> service or change the MAX_JOBS_SUBMITTED limit.
>
>       You probably shouldn't be setting MAX_JOBS_SUBMITTED at all.  It's a cap on the total number of clusters a schedd is willing to have dealt with for its entire life.  What are you trying to accomplish?


This is incorrect. MAX_JOBS_SUBMITTED is a cap on the number of jobs that can be queued at any given time.

 - Jaime
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/