Re: [HTCondor-users] Condor-CE Configuration for SLURM Jobs with Singularity Containers

[John M Knoeller <johnkn@xxxxxxxxxxx>]

I have limited experience with Singularity and SLURM, but my understanding is  you don't need to do anything special in the CE to make this work.  So, to address your questions

1. Are there specific Condor-CE configuration parameters needed to pass Singularity container requirements through to SLURM?

no

2. Should the submit file use universe = grid or universe = vanilla when submitting to the CE?

vanilla.  The CE will change the universe as needed to route the job to Slurm

3. How should I specify the Singularity image in the submit file so it's properly passed to SLURM? (I'm currently using +SingularityImage)

That should work, but I would recommend that you use the new first class submit keyword container_image instead. 

4. Do I need specific ClassAd attributes in the job router configuration to handle Singularity parameters?

no

5. Are there any SLURM-specific settings in the CE route configuration that need to be set for Singularity job

None that I know of.

If anyone reading has details to add, I invite them to weight in.

-tj

---

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Kamrul Shaker <Kamrul.Shaker@xxxxxxxxxxxx>
Sent: Wednesday, December 10, 2025 10:53 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] Condor-CE Configuration for SLURM Jobs with Singularity Containers

Hello HTCondor Community,

I'm setting up a workflow where I need to submit jobs from a Condor-CE client that will be routed to a SLURM cluster, with jobs running inside Singularity containers on the SLURM worker nodes.

Current Setup:

• HTCondor-CE configured with SLURM batch system integration
• SLURM cluster with Singularity support on worker nodes
• Basic job routing configured using JOB_ROUTER_ROUTE_Slurm_Cluster

What I Want to Achieve:

1. Submit jobs from a Condor-CE client
2. Jobs routed through Condor-CE to SLURM
3. Jobs execute within Singularity containers on SLURM worker nodes
4. Container image path specified in the submit file (e.g., from CVMFS)

My Questions:

1. Are there specific Condor-CE configuration parameters needed to pass Singularity container requirements through to SLURM?
2. Should the submit file use universe = grid or universe = vanilla when submitting to the CE?
3. How should I specify the Singularity image in the submit file so it's properly passed to SLURM? (I'm currently using +SingularityImage)
4. Do I need specific ClassAd attributes in the job router configuration to handle Singularity parameters?
5. Are there any SLURM-specific settings in the CE route configuration that need to be set for Singularity jobs?

Current Job Route Configuration:

JOB_ROUTER_ROUTE_Slurm_Cluster @=jrt
  TargetUniverse = 9
  GridResource = "batch slurm"
@jrt
JOB_ROUTER_ROUTE_NAMES = Slurm_Cluster

Any guidance on best practices for this workflow or pointers to relevant documentation would be greatly appreciated.

Best Regards,

Kamrul Shaker

Cloud and High-Throughput Computing Specialist â IT4 (AESES)
Deanâs Office, Faculty of Science
University of Manitoba
ð Kamrul.Shaker@xxxxxxxxxxxx| ð +1 431 335 3508
ð Winnipeg, MB, Canada
ð www.umanitoba.ca