Re: [HTCondor-users] HTCondor within Slurm?

[Paul Hopkins <HopkinsP@xxxxxxxxxxxxx>]

Hi Steffan,

This is entirely possible and a good idea. There are a few of ways you can go about it. Here is a paper that explains how they user Slurm and HTCondor via the grid universe [1].

We have run the LIGO HTCondor Cluster at Cardiff on top of a PBSPro cluster for several years. We used a script to watch the condor_q queue, and when there were idle jobs submitted a job to PBSPro to start a "glidein" HTCondor startd from an installation in a shared NFS file-system. The HTCondor startd actually ran as root so that the node could be shared by multiple people and jobs, but you could submit Slurm jobs as individual users and then restrict the startd to only start their jobs if you do not have that power.

I expect that there are other possibilities as well.

Paul

[1] https://iopscience.iop.org/article/10.1088/1742-6596/898/8/082014/meta

On Sat, 13 Jul 2019 at 16:05, Steffen Grunewald <steffen.grunewald@xxxxxxxxxx> wrote:

Hello all,

I've been asked to install HTCondor on a HPC cluster running Slurm.
While this sounds crazy to me, I might just be ignorant, so I'd like
to ask here before denying the request - has it been done somewhere
else, for whichever reason, and if you did it, would you like to
share your insights?

(We have a HPC setup using Slurm, and a HTC setup using HTCondor.
Both are kept separate, because there's only a partial overlap of
users, and filesystems are shared with a common webserver only.)

Thanks,
 Steffen

--
Steffen Grunewald, Cluster Administrator
Max Planck Institute for Gravitational Physics (Albert Einstein Institute)
Am MÃhlenberg 1 * D-14476 Potsdam-Golm * Germany
~~~
Fon: +49-331-567 7274
Mail: steffen.grunewald(at)aei.mpg.de
~~~
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://eur03.safelinks.protection.outlook.com/?url="">

The archives can be found at:
https://eur03.safelinks.protection.outlook.com/?url="">

--

Paul Hopkins
Computational Infrastructure Scientist

Cardiff University | Prifysgol Caerdydd

+44 (0) 29 225 10043