[HTCondor-users] Condor-CE Configuration for SLURM Jobs with Singularity Containers

[Kamrul Shaker <Kamrul.Shaker@xxxxxxxxxxxx>]

Hello HTCondor Community,

I'm setting up a workflow where I need to submit jobs from a Condor-CE client that will be routed to a SLURM cluster, with jobs running inside Singularity containers on the SLURM worker nodes.

Current Setup:

• HTCondor-CE configured with SLURM batch system integration
• SLURM cluster with Singularity support on worker nodes
• Basic job routing configured using JOB_ROUTER_ROUTE_Slurm_Cluster

What I Want to Achieve:

1. Submit jobs from a Condor-CE client
2. Jobs routed through Condor-CE to SLURM
3. Jobs execute within Singularity containers on SLURM worker nodes
4. Container image path specified in the submit file (e.g., from CVMFS)

My Questions:

1. Are there specific Condor-CE configuration parameters needed to pass Singularity container requirements through to SLURM?
2. Should the submit file use universe = grid or universe = vanilla when submitting to the CE?
3. How should I specify the Singularity image in the submit file so it's properly passed to SLURM? (I'm currently using +SingularityImage)
4. Do I need specific ClassAd attributes in the job router configuration to handle Singularity parameters?
5. Are there any SLURM-specific settings in the CE route configuration that need to be set for Singularity jobs?

Current Job Route Configuration:

JOB_ROUTER_ROUTE_Slurm_Cluster @=jrt
  TargetUniverse = 9
  GridResource = "batch slurm"
@jrt
JOB_ROUTER_ROUTE_NAMES = Slurm_Cluster

Any guidance on best practices for this workflow or pointers to relevant documentation would be greatly appreciated.

Best Regards,

Kamrul Shaker

Cloud and High-Throughput Computing Specialist â IT4 (AESES)
Deanâs Office, Faculty of Science
University of Manitoba
ð Kamrul.Shaker@xxxxxxxxxxxx| ð +1 431 335 3508
ð Winnipeg, MB, Canada
ð www.umanitoba.ca