Re: [HTCondor-users] token jobs not being routed by HTCondor-CE

[Thomas Hartmann <thomas.hartmann@xxxxxxx>]

Hi Stefano,

how does your token mapping look like? ð

Just a suspicion, but maybe the token subject is mapped to another user 
than the X509 mapped user and the requirement
  REQUIREMENTS StringListMember(Owner, "dteam007|dteam026|cmssgm017","|")
does not get triggered?

Cheers,
  Thomas

On 27/03/2023 22.50, Stefano Dal Pra wrote:
> Hello to all,
>
> htcondor-ce-5.1.6 + condor-9.0.17 Here.
>
> I'm having problems with HTCondor-CE not routing jobs submitted with iam 
> token [1]. The same routing rule [2] or [3] working with GSI does not 
> work with tokens.
> More notes in [4].
>
> USING GSI
> #This works
> [sdalpra@ui-htc CE5]$ export 
> _condor_SEC_CLIENT_AUTHENTICATION_METHODS=GSI ; condor_submit -pool 
> ce07-htc.cr.cnaf.infn.it:9619 -remote ce07-htc.cr.cnaf.infn.it 
> ce_gsi308.sub
> Submitting job(s).
> 1 job(s) submitted to cluster 3250129.
>
> #the job is routed and submitted to condor; note the local user 
> (dteam026), that is mapped by argus
> [root@ce07-htc ~]# condor_ce_q 3250129. -af:j owner routedtojobid
> 3250129.0 dteam026 4991835.0
>
> USING SCITOKENS
> #This does not work
> [sdalpra@ui-htc CE5]$ export 
> _condor_SEC_CLIENT_AUTHENTICATION_METHODS=SCITOKENS ; condor_submit 
> -pool ce07-htc.cr.cnaf.infn.it:9619 -remote ce07-htc.cr.cnaf.infn.it 
> ce_scitok308.sub
> Submitting job(s).
> 1 job(s) submitted to cluster 3250138.
>
> #the job is never routed. Note that the REQUIREMENTS expression 
> evaluates to true.
> [root@ce07-htc ~]# condor_ce_q 3250138. -af:j owner routedtojobid 
> 'StringListMember(Owner, "dteam007|dteam026|cmssgm017","|")'
> 3250138.0 dteam007 undefined true
>
>
> [1] The token being used
> [sdalpra@ui-htc CE5]$ cat Â$BEARER_TOKEN_FILE|jwt.py -v
> {
>  Â"alg": "RS256",
>  Â"kid": "rsa1"
> }
> {
>  Â"sub": "9662c0b5-31a1-4478-963e-bdf3783232ed",
>  Â"iss": "https://wlcg.cloud.cnaf.infn.it/";,
>  Â"wlcg.groups": [
>  ÂÂÂ"/wlcg",
>  ÂÂÂ"/wlcg/pilots",
>  ÂÂÂ"/wlcg/xfers"
>  Â],
>  Â"wlcg.ver": "1.0",
>  Â"jti": "4270f069-81d9-48fb-88ef-817a83b98c6a",
>  Â"exp": 1679943559,
>  Â"iat": 1679939959,
>  Â"client_id": "ad852b22-e517-44a4-99e8-7c0660f878a1",
>  Â"scope": "openid compute.create profile compute.read storage.read:/ 
> compute.modify eduperson_entitlement wlcg storage.create:/ 
> offline_access compute.cancel eduperson
> _scoped_affiliation storage.modify:/ email wlcg.groups",
>  Â"nbf": 1679939959,
>  Â"aud": "https://wlcg.cern.ch/jwt/v1/any";
> }
> exp: Mon Mar 27 20:59:19 2023
>
> [2],[3] Jobrouter rules
>
> JOB_ROUTER_ROUTE_routestsci @=jrt
> REQUIREMENTS StringListMember(Owner, "dteam007|dteam026|cmssgm017","|")
>  Â UNIVERSE VANILLA
> SET Requirements (TARGET.t1_group=?= "myfancygroup")
>  ÂÂSET RequestMemory 400
>  ÂÂSET MaxJobs 5
>  ÂÂSET MaxIdleJobs 10
> @jrt
>
> JOB_ROUTER_ROUTE_routestgsi @=jrt
> REQUIREMENTS (x509UserProxyVOName== "dteam") || (AuthTokenIssuer =?= 
> "https://wlcg.cloud.cnaf.infn.it/"&&; AuthTokenSubject =?= 
> "9662c0b5-31a1-4478-963e-bdf3783232ed")
>  ÂUNIVERSE VANILLA
> SET Requirements (TARGET.t1_group=?= "testgroup")
> @jrt
>
> JOB_ROUTER_ROUTE_NAMES= routestsci routestgsi $(JOB_ROUTER_ROUTE_NAMES)
>
> [4] Notes
>
> - scitoken is "partially" valid as the mapping to the local user succeeds.
> - the REQUIREMENTS expression matches with the condor-ce job, i.e.
>  ÂÂÂ condor_ce_q <jobid> -af StringListMember(Owner, 
> "dteam007|dteam026|cmssgm017","|")
>  Â returns True.
> - These rules used to work as far as i know. More complex REQUIREMENTS 
> expressions where successfully used with tokens.
> - I checked rule [2] against a condor-ce at another site where a 
> colleague accepted to test it; the result is the same: using GSI the job 
> is routed, using SCITOKENS it is not.
> - I find nothing useful in the condor-ce logs:
>
> [root@ce07-htc ~]# grep 3250492. /var/log/condor-ce/*Log
> /var/log/condor-ce/AuditLog:03/27/23 21:54:54 (cid:18395186) (D_AUDIT) 
> Submitting new job 3250492.0
> /var/log/condor-ce/AuditLog:03/27/23 21:54:54 (cid:18395188) (D_AUDIT) 
> Transferring files for jobs 3250492.0
> /var/log/condor-ce/SchedLog:03/27/23 21:54:55 (D_ALWAYS) Job 3250492.0 
> released from hold: Data files spooled
>
> Also at maximum verbosity nothing is found in the JobRouterLog.
> I'm out of ideas now. Any hint to find out what's wrong?
> Thanks
> Stefano
>
>
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature