Re: [HTCondor-users] condor_ce_trace authz with tokens?

[Thomas Hartmann <thomas.hartmann@xxxxxxx>]

Hi all,

I finally managed to get some time to learn how to get a token [1]
(thanks to Stefano & Francesco ð).

However, submission authz fails, when I try to use a token from my
subject for trace jobs. In principle, I have prepared the token with
authz only SciTokens enabled [2]. However, the trace jobs still fail
with [3]. AFAIS the token schema should be OK (as "wlcg.ver": "1.0").
Unfortunately, I do not find much more in the logs (FULLDEBUG) what in
the end broke the authorization?

Maybe somebody has a clue for me?

Cheers and thanks,
  Thomas


[1]
> jq -R 'split(".") | .[0],.[1] | @base64d | fromjson' <<< $(cat
$BEARER_TOKEN_FILE)
{
  "kid": "rsa1",
  "alg": "RS256"
}
{
  "wlcg.ver": "1.0",
  "sub": "532fbeea-6f54-4d01-801d-f678239f0637",
  "aud": "https://wlcg.cern.ch/jwt/v1/any";,
  "nbf": 1646055738,
  "scope": "compute.read compute.cancel compute.modify
wlcg.groups:/cms/pilot compute.create",
  "iss": "https://wlcg.cloud.cnaf.infn.it/";,
  "exp": 1646059338,
  "iat": 1646055738,
  "jti": "59212c2a-e929-41ba-834c-3fd19de14c37",
  "client_id": "532fbeea-6f54-4d01-801d-f678239f0637"
}


[2.a]
> export BEARER_TOKEN_FILE=/tmp/bt_u$(id -u)
> export _condor_SEC_CLIENT_AUTHENTICATION_METHODS=SCITOKENS

[2.b]
> /etc/condor-ce/mapfiles.d/99_11_TEST_TOKENS_mapping.conf
SCITOKENS
/^https:\/\/wlcg.cloud.cnaf.infn.it\/,532fbeea-6f54-4d01-801d-f678239f0637/
desyusr000

[2.c]
SCHEDD.SEC_READ_AUTHENTICATION_METHODS = FS,TOKEN,SCITOKENS,GSI,SSL
SCHEDD.SEC_WRITE_AUTHENTICATION_METHODS = FS,TOKEN,SCITOKENS,GSI,SSL
SEC_CLIENT_AUTHENTICATION_METHODS = FS, TOKEN, SCITOKENS, GSI, SSL

[2.d]
> /var/log/condor-ce/MasterLog
02/28/22 15:05:43 Reading mapfile
/etc/condor-ce/mapfiles.d/99_11_TEST_TOKENS_mapping.conf
02/28/22 15:05:43 MapFile: Canonicalization File: method='SCITOKENS'
principal='^https://wlcg.cloud.cnaf.infn.it/,532fbeea-6f54-4d01-801d-f678239f0637'
canonicalization='desyusr000'



[3.a]
> condor_ce_trace  --debug grid-htcondorce-dev.desy.de
...
********************
02/28/22 14:56:55 SharedPortClient: sent connection request to daemon at
<131.169.223.131:9619> for shared port id schedd_4084278_c4f6
02/28/22 14:56:55 Looking for token in file /tmp/bt_u14053
02/28/22 14:56:55 SECMAN: required authentication with daemon at
<131.169.223.131:9619> failed, so aborting command DC_SEC_QUERY.
********************************************************************************
2022-02-28 14:56:55 ERROR: WRITE access failed for scheduler daemon at
<131.169.223.131:9619?addrs=131.169.223.131-9619+[2001-638-700-10df--
1-83]-9619&alias=grid-htcondorce-dev.desy.de&noUDP&sock=schedd_4084278_c4f6>.
WRITE failed!
AUTHENTICATE:1003:Failed to authenticate with any method
AUTHENTICATE:1004:Failed to authenticate using SCITOKENS

[3.b]
> /var/log/condor-ce/SchedLog
...
02/28/22 14:56:55 DC_AUTHENTICATE: required authentication of
131.169.223.90 failed: AUTHENTICATE:1003:Failed to authenticate with any
method|AUTHENTICATE:1004:Failed to authenticate using SCITOKENS
02/28/22 14:56:55 DC_AUTHENTICATE: required authentication of
131.169.223.90 failed: AUTHENTICATE:1003:Failed to authenticate with any
method|AUTHENTICATE:1004:Failed to authenticate using SCITOKENS




On 31/01/2022 16.00, Brian Lin wrote:
> Hi all,
> 
> condor_ce_trace and condor_ce_run should respect the BEARER_TOKEN_FILE
> environment variable. If you're submitting from the CE host itself, you
> should prepend your command with
> '_condor_SEC_CLIENT_AUTHENTICATION_METHODS=SCTIOKENS' to skip local FS
> authentication.
> 
> - Brian
> 
> On 1/31/22 06:56, Stefano Dal Pra wrote:
>> Hello Thomas,
>>
>> I run my tests with a jwt like this one:
>>
>> [sdalpra@ui-htc ~]$ jwt.py -v mytoken ÂÂÂÂÂÂ
>> {
>> Â"alg": "RS256", Â
>> Â"kid": "rsa1"
>> }
>> {
>> Â"sub": "9662c0b5-31a1-4478-963e-bdf3783232ed", Â
>> Â"iss": "https://wlcg.cloud.cnaf.infn.it/";, Â
>> Â"wlcg.groups": [
>> ÂÂÂ"/wlcg", Â
>> ÂÂÂ"/wlcg/pilots", Â
>> ÂÂÂ"/wlcg/xfers"
>> Â], Â
>> Â"wlcg.ver": "1.0", Â
>> Â"jti": "1af5e1a5-c1e6-431f-a7de-d9f2c6cfcb29", Â
>> Â"exp": 1642177529, Â
>> Â"iat": 1642173929, Â
>> Â"client_id": "ad852b22-e517-44a4-99e8-7c0660f878a1", Â
>> Â"scope": "openid compute.create profile compute.read storage.read:/
>> compute.modify eduperson_entitlement wlcg storage.create:/
>> offline_access compute
>> .cancel eduperson_scoped_affiliation storage.modify:/ email
>> wlcg.groups", Â
>> Â"nbf": 1642173929, Â
>> Â"aud": "https://wlcg.cern.ch/jwt/v1/any";
>> }
>> exp: Fri Jan 14 17:25:29 2022
>>
>> I usually set:
>> export BEARER_TOKEN_FILE=/tmp/bt_u`id -u`
>> and put the scitoken there (chmod 600).
>>
>> Then i submit jobs to the CE this way:
>> [sdalpra@ui-htc ~]$ export
>> _condor_SEC_CLIENT_AUTHENTICATION_METHODS=SCITOKENS ; condor_submit
>> -pool ce01t-htc.cr.cnaf.infn.it:9619 -remote ce01t-htc.c
>> r.cnaf.infn.it ce_scitok308.sub
>>
>> The submit file looks like:
>>
>> [sdalpra@ui-htc CE5]$ cat ce_scitok308.sub Â
>> universe = vanilla
>> use_scitokens = true
>> +Owner = undefined
>> [...]
>>
>> The mapfile entry to match my user is:
>>
>> [root@ce01t-htc ~]# grep 9662c0b5-31a1-4478-963e-bdf3783232ed
>> /etc/condor-ce/mapfiles.d/10-scitokens.conf
>> SCITOKENS
>> /^https:\/\/wlcg\.cloud\.cnaf\.infn\.it\/,9662c0b5-31a1-4478-963e-bdf3783232ed/
>> dteam001
>>
>>
>> And I have this entry in the jobrouter; the EVALSET statements are
>> tests of mine in order to set subgroup fairshare based on the
>> wlcg.groups content (currently i have no idea whether this will be
>> actually used not).
>>
>> JOB_ROUTER_ROUTE_dteam001 @=jrt
>> ÂREQUIREMENTS (x509UserProxyVoName =?= "dteam") || (AuthTokenSubject
>> =?= "9662c0b5-31a1-4478-963e-bdf3783232ed")
>> ÂUNIVERSE VANILLA
>> ÂCOPY AuthTokenGroups MyGroup
>> ÂEVALSET TokenGroup strcat(Owner,split(MyGroup,",")[1])
>> ÂEVALSET MyAcctGroup UserMap("AssignAccountingGroup",TokenGroup)
>> Â SET Mytest 100
>> @jrt
>>
>>
>> Finally this is the script i use to decode jwt tokens. Hope this helps
>> Stefano
>>
>> jwt.py (use at your own risk! :) )
>>
>> #!/usr/bin/env python
>>
>> import os, sys, time
>> import base64
>> import json
>>
>> pad = lambda s : s + '='*(len(s)%4)
>> L = sys.argv[1:]
>>
>> x = L and L.pop(0) or '-'
>> verb = x == '-v'
>> if verb:
>> ÂÂÂfn = L and L.pop(0) or '-'
>> else:
>> ÂÂÂfn = (os.path.isfile(x) and x) or '-'
>>
>> f = fn == '-' and sys.stdin or open(fn,'r')
>> ÂÂÂÂ
>> s = f.read()
>> H,B,S = s.split('.')
>> htok = json.loads(base64.urlsafe_b64decode(pad(H)))
>> btok = json.loads(base64.urlsafe_b64decode(pad(B)))
>> print(json.dumps(htok, indent = 2))
>> print(json.dumps(btok, indent = 2))
>> f.close()
>> if verb:
>> ÂÂÂprint('exp: ' + time.ctime(btok.get('exp',0)))
>>
>>
>>
>> On 31/01/22 13:37, Thomas Hartmann wrote:
>>> Hi all,
>>>
>>> is it possible to use a scitoken/wlcg-token to authorize a
>>> `condor_ce_trace/run` debug job (with condor-9.0.8)?
>>>
>>> Cheers,
>>>   Thomas
>>>
>>>
>>>
>>> [doc]
>>> https://htcondor.com/htcondor-ce/v5/troubleshooting/debugging-tools/
>>> https://htcondor.com/htcondor-ce/v5/remote-job-submission/
>>> refers only to X509 grid proxies
>>>
>>> _______________________________________________
>>> HTCondor-users mailing list
>>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
>>> subject: Unsubscribe
>>> You can also unsubscribe by visiting
>>> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>>>
>>> The archives can be found at:
>>> https://lists.cs.wisc.edu/archive/htcondor-users/
>>
>>
>> _______________________________________________
>> HTCondor-users mailing list
>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
>> subject: Unsubscribe
>> You can also unsubscribe by visiting
>> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>>
>> The archives can be found at:
>> https://lists.cs.wisc.edu/archive/htcondor-users/
>

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature