Re: [HTCondor-users] HTCondor and private Docker registries

[Ian McEwen <mian@xxxxxxxxxxx>]

Hi; a member of the HTCondor team might have some recommendations on the
HTCondor side of things, but as far as the docker side of things I have
a couple suggestions that might help you:

1.) If you're running all jobs as one or a small number of users on the
execution nodes (e.g. if you're using SLOTN_USER and
STARTED_ALLOW_RUNAS_OWNER=False, or if you're using an intermediate
process of some sort which submits everything with the same Owner), you
may be able to set up a username and password as part of your execution
node setup; the `docker login` command puts the authentication
information on disk in the user's home directory and HTCondor forks and
uses the docker command-line tools, so if the user it runs docker as is
already logged in, it should be able to pull the image if needed.

2.) If you're fine with the condor nodes being able to pull any image
*without* authentication at all (which would let any user of your condor
cluster pull images from the private registry, which might or might not
be fine for your case), you could use basic auth implemented somewhere
other than the native basic auth, e.g. with nginx:
https://docs.docker.com/registry/recipes/nginx/ . In that case, you
could use the IP-address based access control in the nginx http_access
module: http://nginx.org/en/docs/http/ngx_http_access_module.html
alongside the auth_basic module, and use the 'satisfy' directive
(http://nginx.org/en/docs/http/ngx_http_core_module.html#satisfy) set to
'any' to allow either the IP-based access or the username/password-based
access.

3.) As a modification of 1, if docker and condor is being run as the
submitting user, you could ask users to first run a vanilla-universe job
running `docker login` with their credentials before trying to run
docker universe jobs with private images. One gotcha with both this and
#1 is that if the image has already been pulled on the system, condor
won't need to contact the remote registry at all, so authentication
won't be checked if other users happen to land on that node (and
notably, changes to the image don't get picked up either). One
workaround for this problem could be to run a job afterwards which
deletes the image and runs `docker logout`, though it could be stymied
by concurrent jobs using the same image; YMMV. (probably this login-run
job-delete image-logout process could also be done as a DAG, an exercise
left for the reader ;) )

Hopefully one of these suggestions is helpful, or someone has some other
ideas.

On Fri, Oct 27, 2017 at 12:27:16PM +0200, Luigi Pertoldi wrote:
> Dear HTCondor and Docker experts,
> 
> Iâm configuring HTCondor 8.4 to run jobs using Docker containers. I cannot put the Docker images on a public hub as they contain non-free software, so I created a Docker registry (with the âregistryâ image distributed by Docker) on my local network, and everything works fine with HTCondor. Now I want to restrict access to this registry for few people of my team, for example with a login user and password, as documented for example here <https://docs.docker.com/registry/deploying/#native-basic-auth>. How can I grant access also to the condor user? Should I implement (if possible) a different authentication method, maybe based on personal certificates?
> 
> Thanks in advance
> +ââââââââââââââââââââââââââââââââââââââââââââââââââ+
> | Luigi Pertoldi - PhD student                     |
> |                                                  |
> | Dipartimento di Fisica e Astronomia "G. Galilei" |
> | Università degli Studi di Padova                 |
> | INFN - Sezione di Padova                         |
> | via Marzolo 8, I-35131 Padova (Italy)            |
> | Room 161                                         |
> |                                                  |
> | phone:  +39 049 8277323                          |
> | e-mail: luigi.pertoldi@xxxxxxxxxx <mailto:luigi.pertoldi@xxxxxxxxxx>                |
> | github: github.com/luigipertoldi <https://github.com/luigipertoldi>                 |
> +ââââââââââââââââââââââââââââââââââââââââââââââââââ+
> 



> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/

Attachment: signature.asc
Description: PGP signature