Re: [HTCondor-users] condor_submit pull policy for container universe vs. docker universe

[Matthew Feickert <matthew.feickert@xxxxxxxx>]

Thanks, Jamie. This is super helpful.

> The docker_pull_policy submit command influences how a Docker container image is retrieved on the EP. It is consulted both when the job universe is âdocker' and when the job universe is âsingularity' using a Docker repository path for the container_image.

So, if I understand this correctly, I should switch this back to using the "docker_pull_policy" with the container universe in my HTCondor submit description files and I should still be able to have this work, correct?

> Docker is the only supported containerization system under the container universe that includes an image cache, so we didnât add a separate 'container_pull_policyâ submit command.

This last sentence surprised me. Does that mean that if I'm using a .sif Apptainer container

```

universe = container
container_image = oras://ghcr.io/uw-madison-dsi/pixi-docker-chtc:hello-pytorch-noble-cuda-12.9-apptainer

```

that container image isn't cached at all, and so is pulled from GitHub Container Registry every time?

Best,

Matthew

--
Matthew Feickert
University of Wisconsin-Madison

Book time to meet with me

---

From: Jaime Frey <jfrey@xxxxxxxxxxx>
Sent: Monday, June 16, 2025 15:19
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Cc: Matthew Feickert <matthew.feickert@xxxxxxxx>
Subject: Re: [HTCondor-users] condor_submit pull policy for container universe vs. docker universe

 

The docker_pull_policy submit command influences how a Docker container image is retrieved on the EP. It is consulted both when the job universe is âdocker' and when the job universe is âsingularity' using a Docker repository path for the container_image.

Docker is the only supported containerization system under the container universe that includes an image cache, so we didnât add a separate 'container_pull_policyâ submit command.

 - Jaime

On Jun 12, 2025, at 4:47âPM, Matthew Feickert via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:

Hi All,

In https://github.com/UW-Madison-DSI/pixi-docker-chtc/pull/17 I am converting (among other things) from using the condor_submit "Docker universe" set of options to the "container universe" set of options. In doing so I noticed that my job raised

```

WARNING: the line 'container_pull_policy = missing' was unused by condor_submit. Is it a typo

```

Checking the condor_submit options docs, for the Docker Universe set of commands

https://htcondor.readthedocs.io/en/latest/man-pages/condor_submit.html#docker_pull_policy

> `docker_pull_policy = < always >`:

> if `docker_pull_policy` is set to _always_, when a docker universe job starts on a execution point, the option "âpull always" will be passed to the docker run command. This only impacts EPs which already have a locally cached version of the image. With this option, docker will always check with the repo to see if the cached version is out of date. This requires more network connectivity, and may cause docker hub to throttle future pull requests. It is generally recommended to never mutate docker image tag name, and avoid needing this option.

but then for the container universe set of commands there is no `container_pull_policy`. There is

https://htcondor.readthedocs.io/en/latest/man-pages/condor_submit.html#transfer_container

> `transfer_container = < True | False >`

> A boolean value that defaults to True. When `false`, sif container images and expanded directories are assumed to be pre-staged on the EP, and HTCondor will not attempt to transfer them.

but that sounds like it might be different(?).

Do you know if this means that there is no way to overwrite the container cache if the container universe is used?

Best,

Matthew

--
Matthew Feickert
University of Wisconsin-Madison

Book time to meet with me

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe

Join us in June at Throughput Computing 25: https://urldefense.com/v3/__https://osg-htc.org/htc25__;!!Mak6IKo!Po7dl7Yc8bx5BSDLs5aGKxVM0IBYKc1CjNnSNgx8jE-NcS2F44Kxl9zmwXDPSOc5sSat7soo583XjpOwwlyw3lLyylIFTA$

The archives can be found at: https://www-auth.cs.wisc.edu/lists/htcondor-users/