Re: [HTCondor-users] v24.0.4 condor_submit only works sometimes

[Maarten Litmaath <Maarten.Litmaath@xxxxxxx>]

Hi Geonmo,

thanks for your contributions to this thread!

The issues have been resolved only today, 

thanks very much to Jaime and Cole!

A summary will soon be sent.

---

From: geonmo@xxxxxxxxxxx <geonmo@xxxxxxxxxxx> on behalf of "류건모" <geonmo@xxxxxxxxxxx>
Sent: Wednesday, February 19, 2025 7:38 AM
To: htcondor-users@xxxxxxxxxxx <htcondor-users@xxxxxxxxxxx>
Cc: Maarten Litmaath <Maarten.Litmaath@xxxxxxx>
Subject: RE: Re: [HTCondor-users] v24.0.4 condor_submit only works sometimes

 

Hello, Maarten.

I'm sure you've already resolved the issue by now, but just in case.

I've been testing submitting jobs using the HTCondor cluster as a VM, and under normal circumstances I don't have this issue.

However, in some cases, FS authentication failed on the local account, so it went to IDTOKENS authentication, and then after authenticating with an account like condor@[DOMAIN], I was able to get an error similar to the one you're seeing.

It appears that there is a record of the IDTOKENS authentication attempt and the error is occurring while performing a WRITE operation to the mapped account information from that authentication.

Because of this issue, the authentication succeeds with FS, but the error appears to be caused by the mapping account issue.

In my case, I completely stopped the condor_schedd daemon with systemctl restart condor, so the mapping cache was not blown, and the issue was not reproduced.

However, I'm wondering if this could be a bug. 

I set SEC_DEFAULT_AUTHENTICATION_METHODS = IDTOKENS,$(SEC_DEFAULT_AUTHENTICATION_METHODS)

value with IDTOKENS enabled first and copied the key used for DAEMON authentication to $HOME/.condor/tokens.d for that account to reproduce the issue.

---------------

[geonmo@lrms condor]$ condor_ping -verbose

Destination:                 local schedd

Remote Version:              $CondorVersion: 24.0.4 2025-02-02 BuildID: 784178 PackageID: 24.0.4-1 GitSHA: c93a1052 $

Local  Version:              $CondorVersion: 24.0.4 2025-02-02 BuildID: 784178 PackageID: 24.0.4-1 GitSHA: c93a1052 $

Session ID:                  lrms:13932:1739946086:0

Instruction:                 WRITE

Command:                     60021

Encryption:                  AES

Integrity:                   AES

Authenticated using:         IDTOKENS

All authentication methods:  TOKEN,TOKEN,FS,TOKEN,TOKEN,KERBEROS,SCITOKENS,SSL

Remote Mapping:              condor@xxxxxxxxxxx

Authorized:                  TRUE

[geonmo@lrms condor]$ ls

condor.log  hostname.err  hostname.out  submit.jds

[geonmo@lrms condor]$ condor_submit submit.jds 

Submitting job(s)

ERROR: Failed to create new User record for condor@xxxxxxxxxxxx

The given user is not allowed to own jobs

----------------

------------------

[geonmo@lrms ~]$ rm ~/.condor/tokens.d/condor\@condor.test 

------------------

------------------

[geonmo@lrms condor]$ condor_submit submit.jds 

Submitting job(s).

ERROR: Failed to commit job submission into the queue.

ERROR: Failed to create new User record for condor@xxxxxxxxxxxx

[geonmo@lrms condor]$ condor_ping

WRITE command using (AES, AES, and FS) succeeded as geonmo@xxxxxxxxxxx to local schedd.

--------------------

Regards,

-- Geonmo

[...]