Re: [HTCondor-users] Htcondor with python ray

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 11/25/2025 11:38 AM, Ram Ban wrote:

Thanks Greg for your reply. Cant really find any stuff to integrate both.

Ray requires isolation in case of dynamic slots, 

    What isolation is missing from dynamic slots that are present in
    static slots?
    
    If given permission by the host OS, for both static and dynamic
    slots HTCondor uses Linux cgroups to isolate memory and CPU and
    swap, can run each dynamic slot with a different UID if desired, and
    also can be configured to isolate process table (pid namespaces) and
    subdirectories on the host (via mount namespaces) .  For even more
    isolation, HTCondor can also run containers or virtual machines in
    dynamic slots. 
    
    regards,
    Todd

and in case of static slots it will be less resource utilisation leading to higher costs for bigger VMs. 

I was hoping to get some resources like Kuberay(combines kubernetes and ray and auto scale)

Thanks for your reply

Raman

On Tue, Nov 25, 2025, 00:16 Greg Thain via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:

On 11/24/25 08:42, Ram Ban wrote:
> Hi all,
>
> I am running htcondor 24 with dynamic allocation of resources on each
> machine, Is there any resources to connect htcondor with python ray,
> to use htcondor as auto scale for ray nodes.
>

Raman:

Many people use many different workflow managements systems (like dask,
snakemake, Task Vine, etc.) on top of HTCondor.  I'm not aware of anyone
using Ray today, but I would not be surprised to learn of someone doing
so.  Launch ray workers from an HTCondor job should be straightforward.

-greg


_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe

The archives can be found at: https://www-auth.cs.wisc.edu/lists/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe

The archives can be found at: https://www-auth.cs.wisc.edu/lists/htcondor-users/ 

-- 
Todd Tannenbaum <tannenba@xxxxxxxxxxx>  University of Wisconsin-Madison
Center for High Throughput Computing    Department of Computer Sciences
Calendar: https://tinyurl.com/yd55mtgd  1205 University Ave.
Phone: (608) 263-7132                   Madison, WI 53706
Personal Zoom Room: https://uwmadison.zoom.us/my/tannenba