Re: [HTCondor-users] Is it possible to flock between a htcondor pool and htcondor-ce pool?

[Pau Ruiz <pau.ruiz@xxxxxxxxxx>]

Hi Todd,

Thanks for your reply.

I understand that the typical configuration about flocking is between two HTCondor Pool, and when one pool is full the jobs are sent to the other one, but we want to do another approximation.

We have this design working right now, an HTCondor pool that vanilla/mpi jobs are sent, and a HTCondor-CE pool that transforms condor jobs to slurm jobs, each pool on different machines and clusters, the thing we want to do is when the HTCondor pool is at full capacity, send the jobs through HTCondor-CE to transform into slurm jobs and execute to another cluster, we look at the documentation, and I found the flocking feature, and i though it's possible to configure like that, but as i said in the previous message I'm having problems with the authentication.

The job transforms from HTCondor to HTCondor-CE is possible due the configuration of GSI authentication certificates but i didn't see that we can use the same with flocking, only using the SCITOKENS method authentication.

For this reason, I want to know if it's possible to do this design with HTCondor, HTCondor-CE and the flocking feature, or if there's another way to do it. In other words, is it possible to do it?

Regards,

Pau

Missatge de Todd Tannenbaum <tannenba@xxxxxxxxxxx> del dia dj., 18 de jul. 2024 a les 23:52:

On 7/18/2024 4:07 AM, Pau Ruiz wrote:

Hello everyone,

I want to know if it's possible to do some flocking between a HTCondor pool and a HTCondor-CE pool?

Hi Pau,

An HTCondor Pool is anchored by a Central Manager. To configure an Access Point (submit node) to send jobs to two different pools (aka "flocking"), you need to point your Access Point at two different Central Managers. The HTCondor-CE isn't involved in a flocking setup.

What is currently sending capacity requests to your HTCondor-CE ?ÂÂ Typically it would be a "factory" associated with a computing consortia such as OSPool, CMS, etc ....

Hope this helps,
Todd

-- 
Todd Tannenbaum <tannenba@xxxxxxxxxxx>  University of Wisconsin-Madison
Center for High Throughput Computing    Department of Computer Sciences
Calendar: https://tinyurl.com/yd55mtgd  1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                   Madison, WI 53706-1685