Re: [HTCondor-users] condor_ssh_to_job/interactive jobs with apptainer

[Carles Acosta <cacosta@xxxxxx>]

Hi Joachim,

About your $HOME issue in Apptainer, I'm not sure, but it may be due to this change introduced in 23.9.6:

Container universe jobs running under Singularity or Apptainer now run with a contained home directory, when HTCondor file transfer is enabled. This means the jobs get the $HOME environment variable set to the scratch directory, and an /etc/passwd entry inside the container with the home directory entry pointed to the same place. (HTCONDOR-2274)

You can avoid this behaviour using Should_transfer_files=No.

Cheers,

Carles

On Fri, 23 May 2025 at 09:07, Joachim Meyer <jmeyer@xxxxxxxxxxxxxxxxxx> wrote:

Hi,

yes, I was referring to $HOME inside the container

As far as I can tell, $HOME is overriden only due to setting "--home"

Right now, in the default setup, $HOME is not mounted, but we do usually add a bind mount to /home, thus making it available.

The current command line:

> /usr/bin/singularity -s exec -S /tmp -S /var/tmp -W /raid/condor/lib/condor/execute/dir_1040058 --pwd /raid/condor/lib/condor/execute/dir_1040058 -B /raid/condor/lib/condor/execute/dir_1040058 --nv -B /scratch -B /home -B /etc/OpenCL/vendors --home /raid/condor/lib/condor/execute/dir_1040058 -C /scratch/gpu_admin/jmeyer/pytorch_24.02-py3.sif run.sh

What seems to work for me:

> /usr/bin/singularity -s exec -S /tmp -S /var/tmp -W /raid/condor/lib/condor/execute/dir_1040058 --pwd /raid/condor/lib/condor/execute/dir_1040058 -B /raid/condor/lib/condor/execute/dir_1040058 --nv -B /scratch -B /home -B /etc/OpenCL/vendors -C /scratch/gpu_admin/jmeyer/pytorch_24.02-py3.sif run.sh

So the question is, does HTCondor have to set the --home arg?

Best,

- Joachim

Am Donnerstag, 22. Mai 2025, 17:22:32 MitteleuropÃische Sommerzeit schrieb Greg Thain via HTCondor-users:

>

> On 5/22/25 6:25 AM, Joachim Meyer wrote:

> >

> > - HOME is overwritten (I assume that is due to trying to workaround

> > this stuff:

> > https://htcondor.readthedocs.io/en/latest/faq/admins/interactive-containers-x11.html

> > <https://urldefense.com/v3/__https://htcondor.readthedocs.io/en/latest/faq/admins/interactive-containers-x11.html__;!!Mak6IKo!MFeYNMdiTzJ3juaRMnR-kuMgUqjHHEjDyOOGQJN9SBP_9ex2XzSHSItMBcw_FpJQZ_tCo8MtbRV7YFXbdDV9pctU$>Â?

> > I really don't care about X11... :/ )

> >

>

> Hi Joachim:

>

> For this issue, are we talking about the HOME environment variable

> *inside* the container? If so, I believe that apptainer itself

> unconditionally sets that.

>

>

> -greg

>
--

Joachim Meyer

HPC-Koordination & Support

UniversitÃt des Saarlandes

FR Informatik | HPC

Postanschrift: Postfach 15 11 50 | 66041 SaarbrÃcken

Besucheranschrift: Campus E1 3 | Raum 4.03

66123 SaarbrÃcken

T: +49 681 302-57522

jmeyer@xxxxxxxxxxxxxxxxxx

www.uni-saarland.de

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe

Join us in June at Throughput Computing 25: https://osg-htc.org/htc25

The archives can be found at: https://www-auth.cs.wisc.edu/lists/htcondor-users/

--

Carles Acosta i Silva

PIC (Port d'Informacià CientÃfica)

Campus UAB, Edifici D

E-08193 Bellaterra, Barcelona

Tel: +34 93 581 33 08

Fax: +34 93 581 41 10

http://www.pic.esÂ

AvÃs - Aviso - Legal Notice: Âhttp://legal.ifae.es