Re: [HTCondor-users] Possible misbehavior when using UNDEFINED OAuth Services

[Jaime Frey <jfrey@xxxxxxxxxxx>]

Hi Alan.

The appropriate way to clear a previous setting of use_oauth_services in the submit description is to set it to an empty value, like this:

use_oauth_services =

That is treated the same as not setting it at all for any following queue statements.

 - Jaime

> On Feb 11, 2026, at 8:55âPM, Alan Malta <alanmalta@xxxxxxxxx> wrote:
> 
> Hi Cole,
> 
> Thank you for looking into this and confirming the undesired behavior.
> I wonder if you have any advice or suggestions on how this issue could
> be avoided? Otherwise, do you think this is a change that could be
> made in HTCondor?
> 
> Best regards,
> Alan.
> 
> On Thu, Feb 5, 2026 at 9:20âAM Cole Bollig via HTCondor-users
> <htcondor-users@xxxxxxxxxxx> wrote:
>> 
>> Hi Alan,
>> 
>> There is some subtleness here in the fact that the Job Description Language (JDL) is not the ClassAd language despite looking very similar, and some commands supporting commands supporting ClassAd expressions/behavior. The JDL is essentially a higher-level language full of commands that condor_submit translates into ClassAds to send to the AP. This is why setting use_oauth_services = UNDEFINED is resulting in the string 'UNDEFINED' and not the ClassAd language UNDEFINED. I am not sure how or if one can set the value to the ClassAd value of UNDEFINED. I will need to converse with others on the team.
>> 
>> -Cole Bollig
>> ________________________________
>> From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Alan Malta <alanmalta@xxxxxxxxx>
>> Sent: Wednesday, February 4, 2026 2:42 PM
>> To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
>> Cc: Alan Malta Rodrigues <alan.malta@xxxxxxx>
>> Subject: [HTCondor-users] Possible misbehavior when using UNDEFINED OAuth Services
>> 
>> Dear HTCondor experts,
>> 
>> we are finalizing token integration into CMS production workflows, and
>> due to some old application versions that need to be supported, we
>> need to instrument the WMS job submission component to either enable
>> or disable OAuth, via:
>> """
>> use_oauth_services = cms
>> """
>> or
>> """
>> use_oauth_services = UNDEFINED
>> """
>> 
>> However, when setting this job classad to UNDEFINED, it looks like it
>> parses it as any other string, instead of the reserved UNDEFINED word.
>> Note that, if I set it to `cms`, it works well - as this is one of the
>> credentials stored in condor credd.
>> 
>> One option to workaround this issue would be to not set
>> `use_oauth_services` at all, but then we hit an issue that has been
>> discussed in another thread, see:
>> https://www-auth.cs.wisc.edu/lists/htcondor-users/2025-November/msg00029.shtml 
>> 
>> Would you have any recommendation on what could be done? Is it an
>> issue that should be resolved in HTCondor?
>> 
>> I am also providing details of my setup and how to reproduce it,
>> please see [1] for more information.
>> 
>> Thank you in advance.
>> Best regards,
>> Alan.
>> 
>> [1]
>> ### job submission
>> xxx@vocmsxxx:tokens $ condor_submit submit3.jdl
>> Submitting job(s)
>> Attempting to get tokens for UNDEFINED
>> htgettoken: Failure getting token from https://urldefense.com/v3/__https://xxx.cern.ch:xxx__;!!Mak6IKo!PTpEvNI2yWShGGcLuvJb6R0TtYwXvIiGcu6J8T9RtWYr2fY70HH12iOneVnPqVkA6Wb2M8vOKtPOCAajE1g$
>> Authentication needed for UNDEFINED
>> Attempting OIDC authentication with https://urldefense.com/v3/__https://xxx.cern.ch:xxx__;!!Mak6IKo!PTpEvNI2yWShGGcLuvJb6R0TtYwXvIiGcu6J8T9RtWYr2fY70HH12iOneVnPqVkA6Wb2M8vOKtPOCAajE1g$
>> htgettoken: Initiating authentication to https://urldefense.com/v3/__https://xxx.cern.ch:xxx__;!!Mak6IKo!PTpEvNI2yWShGGcLuvJb6R0TtYwXvIiGcu6J8T9RtWYr2fY70HH12iOneVnPqVkA6Wb2M8vOKtPOCAajE1g$
>> failed: HTTPError: Forbidden: permission denied
>> condor_vault_storer: htgettoken failed
>> Failed to process job credential requests (1):
>> 'process_job_credentials(): invoking '/usr/bin/condor_vault_storer'
>> failed: 0 (Success)
>> '; BAILING OUT.
>> 
>> ### condor version
>> xxx@vocmsxxx:tokens $ condor_version
>> $CondorVersion: 24.0.6 2025-03-05 BuildID: 790851 PackageID: 24.0.6-1 $
>> $CondorPlatform: x86_64_AlmaLinux9 $
>> 
>> ### submit file
>> xxx@vocmsxxx:tokens $ cat submit3.jdl
>> Universe = vanilla
>> executable = test.sh
>> should_transfer_files = YES
>> request_memory = 1 GB
>> request_cpus = 1
>> request_disk = 1 GB
>> # Where to submit - cms specific configuration
>> requirements = stringListMember(GLIDEIN_CMSSite,DESIRED_Sites)
>> +DESIRED_Sites = "T1_US_FNAL,T2_CH_CERN"
>> accounting_group = analysis
>> +AccountGroup = analysis
>> Output = test.$(Cluster)-$(Process).out
>> Error = test.$(Cluster)-$(Process).err
>> Log = test.$(Cluster).log
>> use_oauth_services = UNDEFINED
>> Queue 1
>> _______________________________________________
>> HTCondor-users mailing list
>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
>> subject: Unsubscribe
>> 
>> The archives can be found at: https://www-auth.cs.wisc.edu/lists/htcondor-users/ 
>> _______________________________________________
>> HTCondor-users mailing list
>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
>> subject: Unsubscribe
>> 
>> The archives can be found at: https://www-auth.cs.wisc.edu/lists/htcondor-users/
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> 
> The archives can be found at: https://www-auth.cs.wisc.edu/lists/htcondor-users/