Re: [HTCondor-users] Submit job from host to htcondor/mini container using python bindings

[Cole Bollig <cabollig@xxxxxxxx>]

Hi Gavin,

Some quick notes/questions:

1. Just so I understand correctly, you are running a mini-condor in a container and trying to place jobs to the system from the host machine outside the container?
1. Is there a reason that you need to place jobs from outside the container?
2. Where does the 99-NO_AUTH.config file come from? I.e. Did you find it online or did someone share it with you?
3. I would recommend using the new python API (import htcondor2) as the original is going to be removed in the future.

-Cole Bollig

---

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Gavin Price <gaprice@xxxxxxx>
Sent: Friday, September 12, 2025 1:51 PM
To: htcondor-users@xxxxxxxxxxx <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] Submit job from host to htcondor/mini container using python bindings

 

Hi all,

I started using HTCondor literally Tuesday so I'm an ultranoob. I'm trying to do something that I naively think should be pretty simple, e.g. run the htcondor/mini container and submit a job from the host using the python client. At this point I've spent hours reading docs, googling, and having AI run me in circles. Here's the basics of what I'm doing:

```

$ docker run -d -v ./docker_compose/htcondor/99-NO_AUTH.config:/etc/condor/config.d/99-NO_AUTH.config -p 9618:9618 htcondor/mini:24.11.2-el9
c49d9b7f6d8845badc940ce539b2890e34b34a826ddb822c0ab3d8f9b2eee703
$ docker exec -it agitated_bhaskara bash
[root@c49d9b7f6d88 /]# cat /etc/condor/config.d/99-NO_AUTH.config
# -------------------------------
# Disable all authentication for testing
SEC_DEFAULT_AUTHENTICATION = NEVER
SEC_DEFAULT_AUTHENTICATION_METHODS = CLAIMTOBE
SEC_DEFAULT_ENCRYPTION = OPTIONAL
SEC_DEFAULT_INTEGRITY = OPTIONAL
ALLOW_WRITE = *
ALLOW_READ  = *
ALLOW_ADMINISTRATOR = *

# -------------------------------
# Network configuration for host access
# Set these to the container's IP reachable from the host
#NETWORK_HOST = 172.18.0.3
#FULL_HOSTNAME = 172.18.0.3

SCHEDD_DEBUG = D_ALL.2
```

Then from the host:

```

In [1]: import htcondor
.venv/lib/python3.12/site-packages/htcondor/__init__.py:49: UserWarning: Neither the environment variable CONDOR_CONFIG, /etc/condor/, /usr/local/etc/, nor ~condor/ contain a condor_config source. Therefore, we are using a null condor_config.
  _warnings.warn(message)

In [2]: htcondor.param.update({
   ...:     "SEC_DEFAULT_AUTHENTICATION": "NEVER",
   ...:     "SEC_DEFAULT_AUTHENTICATION_METHODS": "CLAIMTOBE",
   ...:     "SEC_DEFAULT_ENCRYPTION": "OPTIONAL",
   ...:     "SEC_DEFAULT_INTEGRITY": "OPTIONAL",
   ...:     "ALLOW_WRITE": "*",
   ...:     "ALLOW_READ": "*",
   ...:     "ALLOW_ADMINISTRATOR": "*",
   ...: })

In [3]: collector = htcondor.Collector("172.17.0.2:9618")

In [4]: schedd_ad = collector.locate(htcondor.DaemonTypes.Schedd)

In [5]: schedd_ad["MyAddress"]
Out[5]: '<172.17.0.2:9618?addrs=172.17.0.2-9618&alias=c49d9b7f6d88&noUDP&sock=schedd_18_eccb>'

In [6]: schedd = htcondor.Schedd(schedd_ad)

In [7]: sub = htcondor.Submit({
   ...:     "executable": "/bin/sleep",
   ...:     "arguments": "30",
   ...:     "output": "/tmp/sleep.out",
   ...:     "error": "/tmp/sleep.err",
   ...:     "log": "/tmp/sleep.log",
   ...: })

In [8]: cluster_id = schedd.submit(sub)
---------------------------------------------------------------------------
HTCondorIOError                           Traceback (most recent call last)
Cell In[8], line 1
----> 1 cluster_id = schedd.submit(sub)

File ~/.venv/lib/python3.12/site-packages/htcondor/_lock.py:70, in add_lock.<locals>.wrapper(*args, **kwargs)
     67 try:
     68     acquired = LOCK.acquire()
---> 70     rv = func(*args, **kwargs)
     72     # if the function returned a context manager,
     73     # create a LockedContext to manage the lock
     74     is_cm = is_context_manager(rv)

HTCondorIOError: Failed to connect to schedd.
```

The SchedLog logs aren't particularly helpful:

```

09/11/25 01:22:45 TransferQueueManager upload 1m I/O load: 0 bytes/s  0.000 disk load  0.000 net load
09/11/25 01:22:45 TransferQueueManager download 1m I/O load: 0 bytes/s  0.000 disk load  0.000 net load
09/11/25 01:32:09 DC_AUTHENTICATE: Unable to reconcile!
```

I assume the last line is the issue, but I'm not sure how to fix it. This post doesn't seem to have a solution: https://www-auth.cs.wisc.edu/lists/htcondor-users/2013-November/msg00016.shtml

I also tried `SCHEDD_DEBUG = D_ALL` but that spammed the heck out of the logs.

I can run condor_submit from inside the container and have the job run.

I've tried way too many variations on the above to list here (some of them were trying localhost vs. an IP address, setting the variables commented out in the mounted config file both in that file and in the python client params, etc). but I'm happy to go into detail if that would help. That being said, I'm hoping that an expert here could just point me at the correct incantation. 

Thanks a bunch, Gavin