On 3/28/2023 5:42 AM, Stefano Dal
Pra wrote:
When using (only) x509 and no token, the job is mapped (by
argus) to dteam026.
StringListMember should work the same with dteam007 or
dteam026
however it only matches with dteam026 (i.e. GSI). and not
with dteam007.
I normally check for issuer and subject in the jobrouter; i
tried with StringListMember to
restrict the check to Owner only.
Hi Stefano -
After changing the route to try StringListMember, did you
remember to issue a "condor_ce_reconfig"
command?Â
For job 3250138.0 below, it sure looks like the owner mapping
from the token worked fine... perhaps this command will give a
clue:
root@host # condor_ce_history -l 3250138.0 | condor_ce_job_router_info -match-jobs -ignore-prior-routing -jobads -
Also see the CE Manual for troubleshooting tips when a job does
not route at URL:
 https://htcondor.com/htcondor-ce/v4/troubleshooting/troubleshooting/#jobs-stay-idle-on-the-ce
Hope the above helps, let us know how it goes, feel free to ask
for more help if you continue to be stuck.
regards,
Todd
Adding a detail on the submit
file used for GSI and SCITOKENS
#submit file for GSI
[sdalpra@ui-htc
CE5]$ cat ce_gsi308.sub
universe = vanilla
use_x509userproxy = true
+Owner = undefined
[...]
[sdalpra@ui-htc
CE5]$ cat ce_scitok308.sub Â
universe = vanilla
use_scitokens = true
+Owner = undefined
Stefano
On 28/03/23 11:56, Thomas Hartmann wrote:
Hi
Stefano,
how does your token mapping look like? ð
Just a suspicion, but maybe the token subject is mapped to
another user than the X509 mapped user and the requirement
 REQUIREMENTS StringListMember(Owner,
"dteam007|dteam026|cmssgm017","|")
does not get triggered?
Cheers,
 Thomas
On 27/03/2023 22.50, Stefano Dal Pra wrote:
Hello to all,
htcondor-ce-5.1.6 + condor-9.0.17 Here.
I'm having problems with HTCondor-CE not routing jobs
submitted with iam token [1]. The same routing rule [2] or
[3] working with GSI does not work with tokens.
More notes in [4].
USING GSI
#This works
[sdalpra@ui-htc CE5]$ export
_condor_SEC_CLIENT_AUTHENTICATION_METHODS=GSI ;
condor_submit -pool ce07-htc.cr.cnaf.infn.it:9619 -remote
ce07-htc.cr.cnaf.infn.it ce_gsi308.sub
Submitting job(s).
1 job(s) submitted to cluster 3250129.
#the job is routed and submitted to condor; note the local
user (dteam026), that is mapped by argus
[root@ce07-htc ~]# condor_ce_q 3250129. -af:j owner
routedtojobid
3250129.0 dteam026 4991835.0
USING SCITOKENS
#This does not work
[sdalpra@ui-htc CE5]$ export
_condor_SEC_CLIENT_AUTHENTICATION_METHODS=SCITOKENS ;
condor_submit -pool ce07-htc.cr.cnaf.infn.it:9619 -remote
ce07-htc.cr.cnaf.infn.it ce_scitok308.sub
Submitting job(s).
1 job(s) submitted to cluster 3250138.
#the job is never routed. Note that the REQUIREMENTS
_expression_ evaluates to true.
[root@ce07-htc ~]# condor_ce_q 3250138. -af:j owner
routedtojobid 'StringListMember(Owner,
"dteam007|dteam026|cmssgm017","|")'
3250138.0 dteam007 undefined true
[1] The token being used
[sdalpra@ui-htc CE5]$ cat Â$BEARER_TOKEN_FILE|jwt.py -v
{
ÂÂ"alg": "RS256",
ÂÂ"kid": "rsa1"
}
{
ÂÂ"sub": "9662c0b5-31a1-4478-963e-bdf3783232ed",
ÂÂ"iss": "https://wlcg.cloud.cnaf.infn.it/",
ÂÂ"wlcg.groups": [
ÂÂÂÂ"/wlcg",
ÂÂÂÂ"/wlcg/pilots",
ÂÂÂÂ"/wlcg/xfers"
ÂÂ],
ÂÂ"wlcg.ver": "1.0",
ÂÂ"jti": "4270f069-81d9-48fb-88ef-817a83b98c6a",
ÂÂ"exp": 1679943559,
ÂÂ"iat": 1679939959,
ÂÂ"client_id": "ad852b22-e517-44a4-99e8-7c0660f878a1",
ÂÂ"scope": "openid compute.create profile compute.read
storage.read:/ compute.modify eduperson_entitlement wlcg
storage.create:/ offline_access compute.cancel eduperson
_scoped_affiliation storage.modify:/ email wlcg.groups",
ÂÂ"nbf": 1679939959,
ÂÂ"aud": "https://wlcg.cern.ch/jwt/v1/any"
}
exp: Mon Mar 27 20:59:19 2023
[2],[3] Jobrouter rules
JOB_ROUTER_ROUTE_routestsci @=jrt
REQUIREMENTS StringListMember(Owner,
"dteam007|dteam026|cmssgm017","|")
ÂÂ UNIVERSE VANILLA
SET Requirements (TARGET.t1_group=?= "myfancygroup")
ÂÂÂSET RequestMemory 400
ÂÂÂSET MaxJobs 5
ÂÂÂSET MaxIdleJobs 10
@jrt
JOB_ROUTER_ROUTE_routestgsi @=jrt
REQUIREMENTS (x509UserProxyVOName== "dteam") ||
(AuthTokenIssuer =?= "https://wlcg.cloud.cnaf.infn.it/"&&
AuthTokenSubject =?=
"9662c0b5-31a1-4478-963e-bdf3783232ed")
ÂÂUNIVERSE VANILLA
SET Requirements (TARGET.t1_group=?= "testgroup")
@jrt
JOB_ROUTER_ROUTE_NAMES= routestsci routestgsi
$(JOB_ROUTER_ROUTE_NAMES)
[4] Notes
- scitoken is "partially" valid as the mapping to the
local user succeeds.
- the REQUIREMENTS _expression_ matches with the condor-ce
job, i.e.
ÂÂÂÂ condor_ce_q <jobid> -af StringListMember(Owner,
"dteam007|dteam026|cmssgm017","|")
ÂÂ returns True.
- These rules used to work as far as i know. More complex
REQUIREMENTS expressions where successfully used with
tokens.
- I checked rule [2] against a condor-ce at another site
where a colleague accepted to test it; the result is the
same: using GSI the job is routed, using SCITOKENS it is
not.
- I find nothing useful in the condor-ce logs:
[root@ce07-htc ~]# grep 3250492. /var/log/condor-ce/*Log
/var/log/condor-ce/AuditLog:03/27/23 21:54:54 (cid:18395186) (D_AUDIT)
Submitting new job 3250492.0
/var/log/condor-ce/AuditLog:03/27/23 21:54:54 (cid:18395188) (D_AUDIT)
Transferring files for jobs 3250492.0
/var/log/condor-ce/SchedLog:03/27/23 21:54:55 (D_ALWAYS)
Job 3250492.0 released from hold: Data files spooled
Also at maximum verbosity nothing is found in the
JobRouterLog.
I'm out of ideas now. Any hint to find out what's wrong?
Thanks
Stefano
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
--
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing Department of Computer Sciences
Calendar: https://tinyurl.com/yd55mtgd 1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132 Madison, WI 53706-1685
![[Computer Systems Lab]](http://www.cs.wisc.edu/pics/csl_logo.gif){kind=logo}