Re: [HTCondor-users] Docker force ignore cache

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 10/5/2016 3:35 PM, Michael Fienen wrote:
> Hi HTCondorians
>
> I am running in docker universe. The image I created required a very
> small change. However, I had already run a job with that image, so it
> was in the cache on most of my workers. If I delete it manually from
> the cache join a single machine and run it again, it downloads the
> new updated version. Is there a way to do this via htcondor? Would be
> nice to instruct HTCondor that I need a fresh pull of the image.

I am not a docker universe expert, but I think that HTCondor's docker 
universe does the equal of a "docker run".  Perhaps it should be doing 
the equal of a docker pull followed by a docker run; seems like that 
would be most intuitive, but of course the bad news would be more hits 
to your hub.

Something I think you could do right now to deal with the above is I 
believe the docker image you specify in your submit file can be 
decorated to force a specific image version, by either tag name or 
digest.  If the version in the cache is not what you specified in your 
submit file, it will not be used, and the correct version should be 
fetched and cached. You do this by just appending ":tagID" or 
"@digestID" to the name of the image in your submit file.

Some examples (shamelessly copied from the docker documentation) appear 
below.

Hope this helps
Todd

Image[:tag]

While not strictly a means of identifying a container, you can specify a 
version of an image you’d like to run the container with by adding 
image[:tag] to the command. For example, docker run ubuntu:14.04.

Image[@digest]

Images using the v2 or later image format have a content-addressable 
identifier called a digest. As long as the input used to generate the 
image is unchanged, the digest value is predictable and referenceable.

The following example runs a container from the alpine image with the 
sha256:9cacb71397b640eca97488cf08582ae4e4068513101088e9f96c9814bfda95e0 
digest:

$ docker run 
alpine@sha256:9cacb71397b640eca97488cf08582ae4e4068513101088e9f96c9814bfda95e0 
date



> Thanks! Mike Fienen USGS Wisconsin Water Science Center
> _______________________________________________ HTCondor-users
> mailing list To unsubscribe, send a message to
> htcondor-users-request@xxxxxxxxxxx with a subject: Unsubscribe You
> can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/


--
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                  Madison, WI 53706-1685