Re: [HTCondor-users] Problems with OpenSSL 3.5.1

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 11/12/2025 8:43 AM, Matthias Schnepf wrote:

Hi all,

We updated openssl via autoupdates this morning to version 3.5.1 on RHEL9. Since then, no new jobs have started. In the ShadowLog on the CE we found a problem with openssl 3.5.1.

    Hi Matthias,
    
    Thank you for reporting this, we will investigate.   Stay tuned. 
     Glad downgrading OpenSSL solved the problem for you in the
    immediate term.... but downgrading security libraries imho is most
    definitely not a long-term fix!  :)
    
    At first blush, it may be related to this change in OpenSSL:
    https://github.com/openssl/openssl/pull/24677
    
    The whole notion of  x509 proxy certs is does not get much love in
    OpenSSL, which is why we have been trying to move the community away
    from x509 proxies to tokens for years....
    
    regards,
    Todd

From our ShadowLog

11/12/25 10:57:11 (pid:522121) (D_ALWAYS) (1121027.0) (522111): Delegation error: C067501B957F0000:error:05800091:x509 certificate routines:X509_REQ_verify_ex:unsupported version:crypto/x509/x_all.c:47:

11/12/25 10:57:11 (pid:522121) (D_ALWAYS) (1121027.0) (522111): Delegation error:
11/12/25 10:57:11 (pid:522121) (D_ALWAYS) (1121027.0) (522111): ReliSock::put_x509_delegation(): delegation failed: X509Credential::Delegate() failed
11/12/25 10:57:11 (pid:522121) (D_ALWAYS) (1121027.0) (522111): Transfer exit info: Success = False | Error[13.0] = '|Error: sending file /var/lib/condor-ce/spool/4393/29/cluster554393.proc29.subproc0/tmp7ght7u55' | Ack = DOWNLOAD | Line = 5580 | Files
= 0 | Retry = True
11/12/25 10:57:11 (pid:522121) (D_ALWAYS) (1121027.0) (522111): DoUpload: SHADOW at 2a00:139c:a:a:86d2:5ee9:4b76:3e82 failed to send file(s) to <[2a00:139c:9:8::b0]:43045>: |Error: sending file /var/lib/condor-ce/spool/4393/29/cluster554393.proc29.subpr
oc0/tmp7ght7u55; STARTER at 2a00:139c:9:8::b0 - |Error: receiving file /tmp/condor_execute/dir_881536/tmp7ght7u55
11/12/25 10:57:11 (pid:522111) (D_ALWAYS) (1121027.0) (522111): File transfer failed (status=0).

Therefore, the proxy delegation does not work, and the proxy file cannot be copied to the WN. I found an issue with openssl that produces the same error message [1]. I'm not sure if this is a new behavior or a bug in the new openssl version. A downgrade to openssl 3.2.2 fixed the problem for us. Our CEs on RHEL8 are not affected since the new openssl version is not available there by default.

Regards,

Matthias

[1] https://github.com/openssl/openssl/issues/28761

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe

The archives can be found at: https://www-auth.cs.wisc.edu/lists/htcondor-users/ 

-- 
Todd Tannenbaum <tannenba@xxxxxxxxxxx>  University of Wisconsin-Madison
Center for High Throughput Computing    Department of Computer Sciences
Calendar: https://tinyurl.com/yd55mtgd  1205 University Ave.
Phone: (608) 263-7132                   Madison, WI 53706
Personal Zoom Room: https://uwmadison.zoom.us/my/tannenba