Re: [HTCondor-devel] Inclusion of additional arguments to docker run

[Brian Candler <b.candler@xxxxxxxxx>]

On 21/12/2015 17:05, htcondor-devel-request@xxxxxxxxxxx wrote:

Date: Tue, 20 Oct 2015 08:53:42 -0500
From: Greg Thain <gthain@xxxxxxxxxxx>
To: htcondor-devel@xxxxxxxxxxx
Subject: Re: [HTCondor-devel] Inclusion of additional arguments to
	docker run
Message-ID: <56264766.6090109@xxxxxxxxxxx>
Content-Type: text/plain; CHARSET=US-ASCII; format=flowed

Brian:

We've now got some code in 8.5 to allow an admin to mount volumes into a 
docker universe job: 
https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=5308

This is a work in progress, but we think this initial implementation 
should be sufficient to get you out of your patch, and back into 
mainline HTCondor.

-Greg

(Aside: this was posted on 20th October but because I get htcondor-devel as a digest, I didn't receive it until 21st December... maybe the digest parameters could be tweaked a bit :-)

Thank you for that Greg. We now have 8.5.1 rolled out and I've been testing it today.

I needed to make condor a member of the docker group - as described at
http://research.cs.wisc.edu/htcondor/manual/v8.5/3_14Setting_Up.html#SECTION004142000000000000000

Also useful clues from here:
https://research.cs.wisc.edu/htcondor/HTCondorWeek2015/presentations/ThainG_Docker.pdf

I was slightly flummoxed by this example on page 36 of the presentation:

universe = docker
executable = ./my_executable
input_files = my_executable

It wouldn't work - but this is because it should have said "transfer_input_files = my_executable"!

The only issue I have remaining is to do with dynamically mounting docker volumes.

I have configured the execution node with this in condor_config.local:

DOCKER_VOLUMES = SSD
DOCKER_VOLUME_DIR_SSD = /media/SSD

and I see in the machine classAd "HasDockerVolumeSSD = true", so that's working.

Now I try the above submit file, modified as follows:

docker_volume_mount = SSD
executable = /bin/sh
transfer_executable = no
arguments = "-c 'ls /media /media/*'"

But I get a warning from the submit:

WARNING: the line 'docker_volume_mount = SSD' was unused by condor_submit. Is it a typo?

And the volume is not mounted:

==> test.err <==
WARNING: Your kernel does not support swap limit capabilities, memory limited without swap.
ls: cannot access /media/*: No such file or directory

==> test.out <==
/media:


I only got the "docker_volume_mount" setting from https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=5308
so perhaps it hasn't been implemented yet?

However it *does* work if I set DOCKER_MOUNT_VOLUMES = SSD in condor_config.local on the execution node. Of course, that makes the volume mounted unconditionally for every condor job.

Thanks,

Brian.