[HTCondor-users] GAHP fails INITIALIZE_FROM_FILE / how to use separate globus libs

["Fischer, Max (SCC)" <max.fischer@xxxxxxx>]

Hi all,

I'm seeing the following errors in the logs when trying grid submission to cream or nordugrid resources:

12/16/16 11:29:39 (pid:3381524) (D_ALWAYS) [3381524] GAHP server pid = 3381561
12/16/16 11:29:39 (pid:3381524) (D_ALWAYS) [3381524] GAHP command 'INITIALIZE_FROM_FILE' failed: -1
12/16/16 11:29:39 (pid:3381524) (D_ALWAYS) [3381524] GAHP: Failed to initialize from file

From what I've found in the GAHP docs, it's expecting my proxy there. Doing this manually fails with the same error code and a message related to Globus:

$ nordugrid_gahp
$GahpVersion: 1.3.0 Dec 13 2016 Nordugrid\ Gahp $
INITIALIZE_FROM_FILE /tmp/x509up_u12080
F -1 Failed\ to\ activate\ Globus modules

Digging through HTCondor-users, I've found a thread [1] attributing such failures to incorrect resolution of Globus libraries. The tread mentions similar errors when the system globus is used instead of condor's own.
Andrew Lahiff asked for a status on this at the beginning of the year [2], but it's been several versions since then.

Anyways, using ldd indeed gives for example
	libglobus_common.so.0 => /usr/lib64/libglobus_common.so.0
instead of
	libglobus_common.so.0 => /usr/lib64/condor/libglobus_common.so.0

Adding a /etc/ld.so.conf.d/ and ldconfig as in [1] still prefers the system globus libraries.
Setting the dependency explicitly via
	export LD_LIBRARY_PATH='/usr/lib64/condor/'
shows the correct libraries with ldd and manually using gahp, INITIALIZE_FROM_FILE succeeds.

Is there a suggested way to have only condor use its own globus libraries? We would really like to keep the separate globus tools.
Can this be set in condor's configuration, without changing it for the rest of the system?

Cheers,
Max

[1] [HTCondor-users] CREAM error: Failed to start gahp
https://lists.cs.wisc.edu/archive/htcondor-users/2015-September/msg00072.shtml

[2] Re: [HTCondor-users] CREAM error: Failed to start gahp
https://lists.cs.wisc.edu/archive/htcondor-users/2016-January/msg00108.shtml

Attachment: smime.p7s
Description: S/MIME cryptographic signature