Re: [Condor-users] run a condor pool out of SGE nodes

[Mats Rynge <rynge@xxxxxxx>]

On 11/14/2011 06:51 PM, Yu Huang wrote:
> So I have a couple of idea to go around this problem. Before I go out
> and try all of them, I want to see if someone has tried (searching
> condor mailing lists turns up nothing or i don't optimize my keywords
> hard enough).
>
>  1. construct a condor pool out of SGE nodes by submitting jobs that run
>     condor master/startd daemon in user space and custom ports. They are
>     all managed by the one of the nodes, which acts as condor pool
>     submission host. As i don't have root privilege whatsoever, I have
>     run everything as normal user.

This is glideins:
http://www.cs.wisc.edu/condor/manual/v7.6/5_4Glidein.html

The glideins are usually submitted over the GRAM interface, but as you 
want to avoid that, we can make a glidein which is submitted via the 
qsub command on the cluster login node.

It sounds like your plan is to have the central manager / submit node be 
one of the compute nodes. Do you have interactive access (ssh for 
example) to the compute nodes? Also consider that the environment is 
more important on the submit node, so it can be easier to have static 
set up for the central manager / submit host and just glidein the startds.

And if I recall correctly, your workload is a mix of memory 
requirements. We might want to have two types of glideins, one with the 
normal mem/N cores configuration and one with a high memory 
configuration. We could also test dynamic slots.

Let me know off-list if you want to meet and setup a test deployment.


>  2. run the condor-g and globus in user space on the SGE cluster
>     submission host. I have to configure the globus's GRAM part to talk
>     to SGE.  And then I submit jobs on the cluster submission host.
>     Again, everything has to be run in user-space.

This would just be a band aid, and probably not work any better than the 
current non-maintained GRAM install.

--
Mats Rynge
USC/ISI - Pegasus Team <http://pegasus.isi.edu>