Mailing List ArchivesAuthenticated access |
![[Computer Systems Lab]](http://www.cs.wisc.edu/pics/csl_logo.gif){kind=logo}
|
Mailing List ArchivesAuthenticated access |
|
With the right combination of configuration and how the HTCondor daemons are started, that can work._______________________________________________
Using accounting_group_user works for all configurations of HTCondor and lets you do the same controlling of scheduling:
Executable = simple.shaccounting_group_user = someuserâqueue
condor_userprio -setceil someuser
Â- Jaime
On Oct 23, 2024, at 5:39âPM, Seung-Jin Sul <ssul@xxxxxxx> wrote:
_______________________________________________Can I set a user account name like
```Executable = simple.sh Arguments = Log = simple.log Output = simple.out Error = simple.error +Owner = "someuser" transfer_executable = True should_transfer_files = yes Queue
```for all the submission script from the user "someuser" and then call
```
condor_userprio -setceil someuser 100
```
?
On Wed, Oct 23, 2024 at 3:29âPM Seung-Jin Sul <ssul@xxxxxxx> wrote:
Hi Tim,
Thank you for the info.Â
So, even though all the jobs are submitted by the same user name, we can use the command like
```
- condor_userprio -setceil xxx 100
```
?
If so, how can we configure multiple users like "xxx"?
For example,
```OWNER Â Â Â BATCH_NAME Â Â Â Â Â Â Â Â Â Â Â Â Â Â ÂSUBMITTED Â DONE Â RUN Â ÂIDLE ÂTOTAL JOB_IDS
condor_pool cromwell_7ed59a84_bLoci        10/23 14:42   Â_   Â1   Â_   Â1 2060972.0
condor_pool cromwell_64d57d94_bLoci        10/23 14:44   Â_   17   Â_   17 2061072.0 ... 2061492.0
condor_pool cromwell_e4f725cd_bLoci        10/23 14:53   Â_  Â110   Â_  Â110 2061479.0 ... 2062485.0
condor_pool cromwell_00abc1c6_bLoci        10/23 14:54   Â_   68   Â_   68 2061524.0 ... 2062509.0```
Here, a user, "xxx" submitted 4 workflows from Cromwell and total 1+17+110+68 tasks (=jobs) are running on the condor pool now.
As I mentioned, all condor jobs' owner is `condor_pool`.In this situation, we want to limit a user, "xxx", to run less than 100 total jobs regardless of the number of workflow submissions from the above level.Â
Any advice will be appreciated.
Regards,
Seung
On Wed, Oct 23, 2024 at 2:48âPM Tim Theisen <tim@xxxxxxxxxxx> wrote:
Hello Seung-jin,
Another way to limit how many cores a single user can utilize is to use "condor_userprio -setceil" command.
...Tim
On 10/23/24 16:08, Seung-Jin Sul wrote:
Hi Jaime,
Thank you for the info.We need to prevent a user from using all the HTCondor compute resources. So it's the latter. We want to control `how many jobs HTCondor will run at a time for each user`.
We will try `accounting_group_user` if it's possible to control tasks submitted by the same user account.
Thank you so much!
Best,ÂSeung-Jin
On Wed, Oct 23, 2024, 1:54âPM Jaime Frey via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:
Do you want to limit the number of jobs submitted to HTCondor for each user, or how many jobs HTCondor will run at a time for each user? You can limit the number of jobs that can be submitted, but additional submission attempts for the user will fail, which may cause Cromwell to think HTCondor is broken._______________________________________________
HTCondor has several mechanisms to control how many of the computing resources can be used by each user, regardless of how many jobs each has submitted. Thus, you can allow Cromwell to submit as many jobs as it wants and ensure one user doesnât monopolize the glideins in SLURM.If all of the jobs are submitted to HTCondor under the same user account, then you can set accounting_group_user in the HTCondor submit description file. This changes the identifier used for scheduling and accounting purposes.
The details of how HTCondor schedules jobs can be found here:Âhttps://htcondor.readthedocs.io/en/latest/admin-manual/cm-configuration.html#configuration-and-operation-of-the-negotiator
In brief, HTCondor attempts to ensure each user gets a âfair shareâ of the compute resources currently available. Jobs are not run in a strict FIFO order. You can configure many of the details of how the resources are divided amongst the users, including giving some users a larger share and setting a max amount that each user can use at a time.
Â- Jaime
On Oct 22, 2024, at 3:34âPM, Seung-Jin Sul <ssul@xxxxxxx> wrote:
Hi,
We are using Cromwell as a workflow engine and HTCondor as a backend. We have tried to throttle the number of jobs submitted to HTCondor from the Cromwell level using HogGroup and HogFactor but it's not working for use as intended.
The compute resource for our HTCondor is configured as a glide-in with SLURM. Thus, our HTCondor is run by a single-user account. So, we're seeking to pass the user's account name with the submission and then control the number of submissions from the HTCondor scheduler side. Can we limit the number of submissions per user from HTCondor?
Thank you!Seung Sul
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________ HTCondor-users mailing list To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a subject: Unsubscribe You can also unsubscribe by visiting https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users The archives can be found at: https://lists.cs.wisc.edu/archive/htcondor-users/-- Tim Theisen (he, him, his) Release Manager HTCondor & Open Science Grid Center for High Throughput Computing Department of Computer Sciences University of Wisconsin - Madison 4261 Computer Sciences and Statistics 1210 W Dayton St Madison, WI 53706-1685 +1 608 265 5736
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/