Mailing List ArchivesAuthenticated access |
![[Computer Systems Lab]](http://www.cs.wisc.edu/pics/csl_logo.gif){kind=logo}
|
Mailing List ArchivesAuthenticated access |
|
Hi Carles,
indeed.. this helps.. although I don't understand the reasoning for the change when enabling file transfers..?
We have users that want their script and inputs transfered in one version and be able to resubmit the same job with changed inputs/scripts.. (I'm not arguing this is great..)
They still write to /home (or /scratch) or read the full dataset from there..
And with Docker, they can just use $HOME (as long as getenv = HOME is set, which we do automatically with submit transforms) - this flow breaks with Apptainer..
I see the code and ticket, but I don't quite understand the relation between HOME env and the 2274 issue.
Best,
- Joachim
Am Freitag, 23. Mai 2025, 09:36:32 MitteleuropÃische Sommerzeit schrieb Carles Acosta:
> Hi Joachim,
>
> About your $HOME issue in Apptainer, I'm not sure, but it may be due to
> this change introduced in 23.9.6:
>
> *Container universe jobs running under Singularity or Apptainer now run
> with a contained home directory, when HTCondor file transfer is enabled.
> This means the jobs get the $HOME environment variable set to the scratch
> directory, and an /etc/passwd entry inside the container with the home
> directory entry pointed to the same place. (HTCONDOR-2274)*
>
> You can avoid this behaviour using Should_transfer_files=No.
>
> Cheers,
>
> Carles
>
>
>
> On Fri, 23 May 2025 at 09:07, Joachim Meyer <jmeyer@xxxxxxxxxxxxxxxxxx>
> wrote:
>
> > Hi,
> >
> > yes, I was referring to $HOME inside the container
> >
> > As far as I can tell, $HOME is overriden only due to setting "--home"
> >
> > Right now, in the default setup, $HOME is not mounted, but we do usually
> > add a bind mount to /home, thus making it available.
> >
> > The current command line:
> >
> > > /usr/bin/singularity -s exec -S /tmp -S /var/tmp -W
> > /raid/condor/lib/condor/execute/dir_1040058 --pwd
> > /raid/condor/lib/condor/execute/dir_1040058 -B
> > /raid/condor/lib/condor/execute/dir_1040058 --nv -B /scratch -B /home -B
> > /etc/OpenCL/vendors --home /raid/condor/lib/condor/execute/dir_1040058 -C
> > /scratch/gpu_admin/jmeyer/pytorch_24.02-py3.sif run.sh
> >
> > What seems to work for me:
> >
> > > /usr/bin/singularity -s exec -S /tmp -S /var/tmp -W
> > /raid/condor/lib/condor/execute/dir_1040058 --pwd
> > /raid/condor/lib/condor/execute/dir_1040058 -B
> > /raid/condor/lib/condor/execute/dir_1040058 --nv -B /scratch -B /home -B
> > /etc/OpenCL/vendors -C /scratch/gpu_admin/jmeyer/pytorch_24.02-py3.sif
> > run.sh
> >
> > So the question is, does HTCondor have to set the --home arg?
> >
> > Best,
> >
> > - Joachim
> >
> > Am Donnerstag, 22. Mai 2025, 17:22:32 MitteleuropÃische Sommerzeit schrieb
> > Greg Thain via HTCondor-users:
> >
> > >
> >
> > > On 5/22/25 6:25 AM, Joachim Meyer wrote:
> >
> > > >
> >
> > > > - HOME is overwritten (I assume that is due to trying to workaround
> >
> > > > this stuff:
> >
> > > >
> > https://htcondor.readthedocs.io/en/latest/faq/admins/interactive-containers-x11.html
> >
> > > > <
> > https://urldefense.com/v3/__https://htcondor.readthedocs.io/en/latest/faq/admins/interactive-containers-x11.html__;!!Mak6IKo!MFeYNMdiTzJ3juaRMnR-kuMgUqjHHEjDyOOGQJN9SBP_9ex2XzSHSItMBcw_FpJQZ_tCo8MtbRV7YFXbdDV9pctU$> ?
> >
> >
> > > > I really don't care about X11... :/ )
> >
> > > >
> >
> > >
> >
> > > Hi Joachim:
> >
> > >
> >
> > > For this issue, are we talking about the HOME environment variable
> >
> > > *inside* the container? If so, I believe that apptainer itself
> >
> > > unconditionally sets that.
> >
> > >
> >
> > >
> >
> > > -greg
> >
> > >
> > --
> >
> > *Joachim Meyer*
> >
> > HPC-Koordination & Support
> >
> > UniversitÃt des Saarlandes
> >
> > *FR Informatik | HPC*
> >
> > Postanschrift: Postfach 15 11 50 | 66041 SaarbrÃcken
> >
> > Besucheranschrift: Campus E1 3 | Raum 4.03
> >
> > 66123 SaarbrÃcken
> >
> > T: +49 681 302-57522
> >
> > jmeyer@xxxxxxxxxxxxxxxxxx
> >
> > www.uni-saarland.de
> > _______________________________________________
> > HTCondor-users mailing list
> > To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with
> > a
> > subject: Unsubscribe
> >
> > Join us in June at Throughput Computing 25: https://osg-htc.org/htc25
> >
> > The archives can be found at:
> > https://www-auth.cs.wisc.edu/lists/htcondor-users/
> >
>
>
>
--
Joachim Meyer
HPC-Koordination & Support
UniversitÃt des Saarlandes
FR Informatik | HPC
Postanschrift: Postfach 15 11 50 | 66041 SaarbrÃcken
Besucheranschrift: Campus E1 3 | Raum 4.03
66123 SaarbrÃcken
T: +49 681 302-57522
jmeyer@xxxxxxxxxxxxxxxxxx
www.uni-saarland.de