Re: [HTCondor-users] Condor unsetting PATH and setting only variable and not environment

[Zach McGrew <mcgrewz@xxxxxxx>]

No worries, Todd. I didn't write a script just to work around this, I just extended scripts that needed access to more than the shell built-ins. This is probably on me and my need to hit every problem with my AWK shaped hammer more than anything. =)

Perhaps the more Condor-like solution would be a job transform to set PATH when it's not explicitly requested in the submitted environment? This way users wouldn't have to know to set it, but if they did they still get what they asked for.


Marco,
To get back to your first question, I suspect this will give you what you want:

environment = "PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin"

Or to automagically pull the value from the submitter's environment when they ran condor_submit:

getenv = PATH

-Zach

________________________________________
From: Todd Tannenbaum <tannenba@xxxxxxxxxxx>
Sent: Monday, June 16, 2025 1:26 PM
To: HTCondor-Users Mail List; Zach McGrew
Subject: Re: [HTCondor-users] Condor unsetting PATH and setting only variable and not environment

You don't often get email from tannenba@xxxxxxxxxxxx Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>

I am sad that Zach felt compelled to write a wrapper script to deal with this.

At least for Bash, the default PATH when not explicitly set is hard-coded within the Bash executable to be

   /user/local/bin:/user/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin

Sounds like folks in this thread feel that HTCondor should do the same when launching a job that does not explicitly specify a PATH ?   Makes sense to me.

regards,
Todd T

On 6/16/2025 1:29 PM, Zach McGrew wrote:

I noticed this a while back as well, I just assumed it was more env variable cleanup/removal for reproducible research purposes. i.e. If you have a different env from the next submitter you get different results, so set the env to the minimum viable env.

I just started setting a PATH in my wrapper scripts and telling my users to do the same:

# Set a sane PATH. HTCondor can eat the default.
export PATH="${PATH:+${PATH}:}/bin:/usr/bin:/sbin:/usr/sbin"



This submission file will print the environment and exit, and my output results are below (err is empty as expected). You'll notice there's no PATH:

executable = /usr/bin/env
output = logs/out
error  = logs/err
log    = logs/condor_log
queue


And results:


_CONDOR_ANCESTOR_14582=16437:1742407785:660241318
_CONDOR_ANCESTOR_16437=614845:1750097658:297724199
_CONDOR_ANCESTOR_614845=614850:1750097658:3023395836
APPTAINER_CACHEDIR=/var/lib/condor/execute/dir_614845
BATCH_SYSTEM=HTCondor
CUBACORES=1
CUDA_VISIBLE_DEVICES=10000
GOMAXPROCS=1
GPU_DEVICE_ORDINAL=10000
JULIA_NUM_THREADS=1
MKL_NUM_THREADS=1
NUMEXPR_NUM_THREADS=1
NVIDIA_VISIBLE_DEVICES=none
OMP_NUM_THREADS=1
OMP_THREAD_LIMIT=1
OPENBLAS_NUM_THREADS=1
PYTHON_CPU_COUNT=1
ROOT_MAX_THREADS=1
SINGULARITY_CACHEDIR=/var/lib/condor/execute/dir_614845
TEMP=/tmp
TF_LOOP_PARALLEL_ITERATIONS=1
TF_NUM_THREADS=1
TMP=/tmp
TMPDIR=/tmp
_CHIRP_DELAYED_UPDATE_PREFIX=Chirp*
_CONDOR_AssignedGPUs=10000
_CONDOR_BIN=/usr/bin
_CONDOR_CHIRP_CONFIG=/var/lib/condor/execute/dir_614845/.chirp.config
_CONDOR_JOB_AD=/var/lib/condor/execute/dir_614845/.job.ad
_CONDOR_JOB_IWD=/cluster/home/mcgrewz/example_submit_scripts/basic
_CONDOR_JOB_PIDS=
_CONDOR_MACHINE_AD=/var/lib/condor/execute/dir_614845/.machine.ad
_CONDOR_SCRATCH_DIR=/var/lib/condor/execute/dir_614845
_CONDOR_SLOT=slot1_1



-Zach

________________________________________
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx><mailto:htcondor-users-bounces@xxxxxxxxxxx> on behalf of Todd L Miller via HTCondor-users <htcondor-users@xxxxxxxxxxx><mailto:htcondor-users@xxxxxxxxxxx>
Sent: Monday, June 16, 2025 11:06 AM
To: Marco Mambelli via HTCondor-users
Cc: Todd L Miller
Subject: Re: [HTCondor-users] Condor unsetting PATH and setting only variable and not environment



When submitting jobs they have PATH=/usr/local/bin:/usr/bin and no PATH
variable in the environment.

1. I don't know where the PATH is coming from
- The PATH of the user the job is running under is different
- This is at the beginning of my job
2. Why is only the variable set and not the environment?
- This causes problems with shell scripts because PATH is undefined in subprocesses (e.g. commands executed in the script)



        I don't understand:

(a) PATH is definitionally the PATH variable in the environment.  You
     can't "have a PATH" if PATH isn't set in the environment.  What are
     you trying to say here?
(b) Given the questions, presumably you meant "When my jobs run," not
     "When submitting jobs"?

        Please send a submit file and shell script that demonstrate the
problem.

-- ToddM

--
Todd Tannenbaum <tannenba@xxxxxxxxxxx><mailto:tannenba@xxxxxxxxxxx>  University of Wisconsin-Madison
Center for High Throughput Computing    Department of Computer Sciences
Calendar: https://tinyurl.com/yd55mtgd<https://tinyurl.com/yd55mtgd>  1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                   Madison, WI 53706-1685