Re: [HTCondor-users] sudo / package installation inside a docker container inside a user job

["Kandes, Martin" <mkandes@xxxxxxxx>]

Gergely,

Dimitri does have a good point. But I understand your problem. I help maintain the Singularity containers we build for our users [1].

One possibility you might consider is using miniconda [2] to install additional packages in  userspace, if the user's HOME directory or some scratch space available to the job is large enough to handle installations of packages like TensorFlow. I actually install miniconda by default in our base containers for this purpose myself. e.g., see [3] [4].

I know this isn't a Docker-native solution. But that's how I might do it with Singularity. 

Marty

[1]

https://github.com/mkandes/naked-singularity

[2]

https://docs.conda.io/en/latest/miniconda.html

[3]

https://github.com/mkandes/naked-singularity/blob/master/definition-files/comet/ubuntu/ubuntu.def

[4]

https://github.com/mkandes/naked-singularity/blob/master/definition-files/comet/ubuntu/ubuntu-cuda.def

---

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Dimitri Maziuk via HTCondor-users <htcondor-users@xxxxxxxxxxx>
Sent: Thursday, June 13, 2019 1:27:50 PM
To: htcondor-users@xxxxxxxxxxx
Cc: Dimitri Maziuk
Subject: Re: [HTCondor-users] sudo / package installation inside a docker container inside a user job

 

On 6/13/19 3:07 PM, Gergely Debreczeni via HTCondor-users wrote:
> python3, tensorflow, gnuplot, etc... nothing special, in principle they could be installed in advance, but each user group has different requirements and we don't want to manage / maintain so many different docker image...

Is reproduce/repet-ability a concern? You may get a different new
version of $foo on every run if you do this; "immutable" is one of the
buzzwords in docker's sales pitch.

--
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu