Re: [Condor-users] Sharing HPC nodes with Condor

[Preston Smith <psmith@xxxxxxxxxx>]

Set a couple things in your condor_config files (s/PBS/SGE/g):

After 6.7.18, you'll have to do the following to let this work:
ENABLE_RUNTIME_CONFIG = TRUE

Then
STARTD_SETTABLE_ATTRS_OWNER = PBSRunning

PBSRunning = False
PURDUE_RCAC_START_NOPBS = ( $(PBSRunning) == False )

START = $(PURDUE_RCAC_START) && $(PURDUE_RCAC_START_NOPBS)


Then in your prologue, when PBS starts a job, mark it unavailable to  
Condor, and run condor_vacate if necessary. Or, Dan Bradley suggested  
to me at Condor Week that it might  be a better option to just  
manually suspend any running jobs when PBS butts in, if your  
environment will allow it.  And then in the epilogue, after PBS is  
done, mark it as available to Condor again.

Prologue:

        if [ -x /opt/condor/bin/condor_config_val ]; then
                /opt/condor/bin/condor_config_val -rset \
                        -startd PBSRunning=True > /dev/null
                /opt/condor/sbin/condor_reconfig -startd > /dev/null
                if ( /opt/condor/bin/condor_status -claimed \
                        -direct $(hostname) 2>/dev/null \
                        | grep -q Machines )
                then
                        /opt/condor/sbin/condor_vacate > /dev/null
                fi
        fi



Epilogue:

                if [ -x /opt/condor/bin/condor_config_val ]; then
                        /opt/condor/bin/condor_config_val -rset \
                                -startd PBSRunning=False > /dev/null
                        /opt/condor/sbin/condor_reconfig -startd > / 
dev/null
                fi


On May 9, 2007, at 2:29 PM, Natarajan, Senthil wrote:

> Hi Preston,
> Thanks for offering your help.
> Yes, we are trying to do something like that.
>
> One of the department has a cluster managed by Sun Grid Engine, and  
> they
> plan to share around 20 nodes to condor, so how can we configure in  
> such
> a way so that, if the nodes are not used by Sun Grid Engine then  
> condor
> can use it. Suppose if the jobs submitted to Sun Grid Engine needs  
> nodes
> have being used by condor, then condor should terminate the job and  
> send
> it back to central manager like if somebody login the machine.
>
> Thanks,
> Senthil
>
> -----Original Message-----
> From: condor-users-bounces@xxxxxxxxxxx
> [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Preston Smith
> Sent: Tuesday, May 08, 2007 9:38 PM
> To: Condor-Users Mail List
> Subject: Re: [Condor-users] Sharing HPC nodes with Condor
>
> Sounds like you're wanting to backfill behind your primary scheduler
> with Condor, correct? We do this on all of our PBS clusters at
> Purdue, I can help with that if that's what you're looking for.
>
> -Preston
>
> On May 8, 2007, at 5:37 PM, Natarajan, Senthil wrote:
>
> > Hi,
> >
> > We are planning to use 20 nodes from a High performance Computing
> > cluster to run condor jobs if there is no jobs running on the HPC
> > nodes.
> >
> > Suppose if HPC users submits jobs and it requires the nodes where
> > condor jobs are running, condor jobs should be terminated and send
> > back to the central manager, like if some real user login the
> > machine. I was wondering how to configure condor to do this. Could
> > you please let me know.
> >
> > Thanks,
> >
> > Senthil
> >
> > _______________________________________________
> > Condor-users mailing list
> > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
> > with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> >
> > The archives can be found at either
> > https://lists.cs.wisc.edu/archive/condor-users/
> > http://www.opencondor.org/spaces/viewmailarchive.action?key=CONDOR
>
>
>
> --
> Preston Smith    <psmith@xxxxxxxxxx>
> Systems Research Engineer
> Rosen Center for Advanced Computing, Purdue University
>
>
>
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx  
> with
> a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at either
> https://lists.cs.wisc.edu/archive/condor-users/
> http://www.opencondor.org/spaces/viewmailarchive.action?key=CONDOR
>
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx  
> with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at either
> https://lists.cs.wisc.edu/archive/condor-users/
> http://www.opencondor.org/spaces/viewmailarchive.action?key=CONDOR

--
Preston Smith  <psmith@xxxxxxxxxx>
Systems Research Engineer
Rosen Center for Advanced Computing, Purdue University