Re: [HTCondor-users] FULL_HOSTNAME issue with multiple NIC

[Bockjoo Kim <bockjoo@xxxxxxxxxxxx>]

I tried MY_FULL_HOSTNAME but it did not work.
Since my htcondor ce needs to submit jobs to pbs,
hostname should be a private hostname.
So, I would like this issue to be fixed.

At the moment, the first five lines of my /etc/condor-ce/condor_config 
looks like:
MY_FULL_HOSTNAME = <public hostname>
CONDOR_HOST = $(MY_FULL_HOSTNAME)
PORT = 9619
COLLECTOR_HOST = $(MY_FULL_HOSTNAME):9619
NETWORK_INTERFACE = <public ip>

After starting the htcondor ce, it is listening to 9618 instead of 9619 
that I specified in the config file:

# netstat -ntpl | grep 96
tcp        0      0 0.0.0.0:9618 0.0.0.0:* LISTEN      32609/condor_collec
tcp        0      0 0.0.0.0:9620 0.0.0.0:* LISTEN      32607/condor_shared

# ps -efL | grep 32609 | grep -v grep
condor   32609 32604 32609  0    1 00:10 ?        00:00:00 
condor_collector -f -port 9619

# lsof -p 32609 | grep CollectorLog
condor_co 32609 condor    3w   REG                9,3   929094 7997332 
/var/log/condor-ce/CollectorLog

If I execute condor_collector -f -port 9619, I get this:
# netstat -ntpl | grep 96
tcp        0      0 0.0.0.0:9618 0.0.0.0:* LISTEN      32609/condor_collec
tcp        0      0 0.0.0.0:9619 0.0.0.0:* LISTEN      584/condor_collecto
tcp        0      0 0.0.0.0:9620 0.0.0.0:* LISTEN      32607/condor_shared

# lsof -p 584 | grep CollectorLog
condor_co 584 condor    3w   REG     9,3   233130  7996135 
/var/log/condor/CollectorLog

It's using the regular condor CollectorLog instead of condor-ce 
CollectorLog.

Our HPC does not want to change the way they manage the cluster and 
leave the hostname
private:
If I manually set up the public hostname and there is a reboot or 
reimaging of the system,
they have to remember to set it up manually.
They want to avoid it
Can something be done about this in the htcondor ce?
Thanks,
Bockjoo

On 4/24/14 3:51 PM, Bockjoo Kim wrote:
> On 4/24/14 3:16 PM, Zachary Miller wrote:
> > On Thu, Apr 24, 2014 at 03:08:08PM -0400, Bockjoo Kim wrote:
> > > On 4/24/14 1:09 PM, Zachary Miller wrote:
> > > > On Wed, Apr 23, 2014 at 08:50:41PM -0400, Bockjoo Kim wrote:
> > > > > Hi Todd,
> > > > > If I add these
> > > > > NETWORK_INTERFACE = <public IP>
> > > > > BIND_ALL_INTERFACES=FALSE
> > > > > PRIVATE_NETWORK_INTERFACE = <private IP>
> > > > > PRIVATE_NETWORK_NAME = <private hostname>
> > > > > to the condor-ce condor_config file which uses the port 9619 as the
> > > > > collector port,
> > > > > collector does not listen to 9619:
> > > > >
> > > > > [root@cms ~]# netstat -ntl | grep 96
> > > > > tcp        0      0 0.0.0.0:9618 0.0.0.0:*                   LISTEN
> > > > > tcp        0      0 128.227.253.206:9620 0.0.0.0:*
> > > > > LISTEN
> > > > Some questions:
> > > >
> > > > Can you add "-p" to netstat to show the pid/process for each of those
> > > > entries?
> > > netstat -ntlp | grep 96
> > > tcp        0      0 0.0.0.0:9618 0.0.0.0:* LISTEN
> > > 14168/condor_collec
> > Hmm. So the collector is running but it seems like it's not honoring 
> > the config
> > you specified.
> It honors in the sense the one running on 9618 is condor #1.
> The collector that needs to run on 9619 is condor #2 ( so called 
> condor-ce in OSG).
> It does not honor in the sense that if the hostname is not the public 
> one, it picks 9618
> and collide with condor #1.
>
>
> >    Can you double check that the condor_config is the one the
> > master is using (and that you don't have a local config file that is 
> > overriding
> > your values)?  Run this:
> >
> >    condor_config_val -master COLLECTOR_HOST
> [root@cms scramrc]# condor_config_val -master COLLECTOR_HOST # condor #1
> cms.rc.ufl.edu
> [root@cms scramrc]# condor_ce_config_val -master COLLECTOR_HOST # 
> condor-ce
> cms.rc.ufl.edu:9619
>
> > > Do you mean I can replace FULL_HOSTNAME with MY_FULL_HOSTNAME and 
> > > define
> > > MY_FULL_HOSTNAME
> > > in the condor config files?
> Ok, I will check this later as I have other more urgent business.
> I am settled for now with the 'hostname cms.rc.ufl.edu' within the 
> machine.
> Thanks,
> Bockjoo
>
> > Yes.
> >
> >
> > Cheers,
> > -zach
> >
> > _______________________________________________
> > HTCondor-users mailing list
> > To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx 
> > with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/htcondor-users/
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx 
> with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/