[HTCondor-users] Dedicated Scheduler Jobs' PATH prefixed by /usr/bin?

[Carsten Aulbert <carsten.aulbert@xxxxxxxxxx>]

Hi all,

we still have some pipelines which need to finish on our old Condor pool 
still running 8.8 but discovered something new (for us at least). This 
only happens when submitting from the Dedicated Scheduler submit host[1].

The simple - trimmed down - submit file

executable = /usr/bin/env

Environment = "PATH=/invalid/path:/usr/local/bin:/usr/bin:/bin"
output = out
error = err
universe = 11
requirements = (Machine =?= "a8101.atlas.local")
queue

results in a job being run but with a major difference in the PATH variable:

condor_history -format "%s" Environment 170983.0
PATH=/invalid/path:/usr/local/bin:/usr/bin:/bin

vs

grep PATH out
PATH=/usr/bin:/invalid/path:/usr/local/bin:/usr/bin:/bin

So, something is prefixing the PATH variable with /usr/bin which causes 
for example havoc to any jobs which rely on a custom python build 
environment they want to call via `#!/usr/bin/env python3` but then the 
first match is the OS's python.

Is there an knob to switch this off or are we beyond the limits of what 
the parallel universe allows us to do?

Cheers

Carsten

[1] We do have flocking enabled but I find it hard to test forcing a job 
to be flocked, therefore no idea if this is something related to the 
Schedd or the startd/starter.

--
Dr. Carsten Aulbert, Max Planck Institute for Gravitational Physics,
CallinstraÃe 38, 30167 Hannover, Germany, Phone +49 511 762 17185

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature