Re: [HTCondor-users] Possible to disable the "docker pull" in htcondor 25?

[Greg Thain <gthain@xxxxxxxxxxx>]

On 10/3/25 15:59, Jordan Corbett-Frank via HTCondor-users wrote:

I recently upgraded to htcondor 25 (by accident, using the convenience script) from htcondor 24 and am now seeing authentication errors because condor is always trying to pull the images from our private registries. I understand I can authenticate the condor user with our private registries, but I'd like to avoid that, if possible:

Hi Jordan:

I'm sorry you are having problems with this.  The idea is that HTCondor wants to be responsible for managing images it pulls down on behalf of jobs, and to ignore other images that may be have been put on the machine outside of it's responsibility.  HTCondor in v25 marks it's own images by "docker tag"-ing them.  In v24 we did this in a file in the filesystem, but this file got removed or overwritten at many sites, leading to problems.

We can add a config knob to avoid this checking if your operational goal is to pre-stage images into docker and only use those.  In the mean time, if you manually create the docker tag to match the image, like in the function DockerAPI::toAnnotatedName, I think that will work for you in the short term.

-greg

1. I'd like to disable the automatic pull so images with floating tags are not automatically updated. By separating the pull from the "run" we have direct control. In htcondor 24, condor could see the images that existed on the host; however in htcondor 25 it does not seem to be able to. It seems like the following code cannot see the already-pulled-down-images. 

ââââââ// See if we've already got this image annotation

ââââââauto it = std::ranges::find(imageInfos, annotatedImageName, &DockerAPI::ImageInfo::imageName);

ââââââif (imageInfos.end() ==  it) {

ââââââââââââreturn PullImage();

2. The way our development-laptop-setup-scripts work right now, condor is installed prior to the developer authenticating with our docker registries and each step has a different mechanism (running an installation script vs. using direnv to automatically login as necessary). I could tell everyone "hey run this command after" but I'd like to keep it turn-key and avoid "sudo" in the .envrc file. 

So to sum it up: why is condor not able to see images that already exist on the host and is it possible to configure it to not pull from the registry?

Thank you, 

Jordan 

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe

The archives can be found at: https://www-auth.cs.wisc.edu/lists/htcondor-users/