Re: [HTCondor-users] Setting up a CE so that pilots (or jobs) advertise back to the CE collector with IDTOKENS

[John M Knoeller <johnkn@xxxxxxxxxxx>]

You can control the tokens on a per-route basis by adding SendIDTokens to the route definition. 

JOB_ROUTER_ROUTE_Fermi @=rt

    ... define route...
   # override JOB_ROUTER_SEND_ROUTE_IDTOKENS for this route
    SendIDTokens = "fermilab2"

@rt

-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Marco Mambelli via HTCondor-users
Sent: Sunday, October 23, 2022 12:13 PM
To: htcondor-users@xxxxxxxxxxx
Cc: Marco Mambelli <marcom@xxxxxxxx>; Nicholas Peregonow <njp@xxxxxxxx>
Subject: [HTCondor-users] Setting up a CE so that pilots (or jobs) advertise back to the CE collector with IDTOKENS

I tried to setup a CE so that pilots (or jobs) advertise back to the CE collector with IDTOKENS.
I followed the instructions in:
https://htcondor.readthedocs.io/en/latest/admin-manual/configuration-macros.html#JOB_ROUTER_CREATE_IDTOKEN_%3CNAME%3E
https://opensciencegrid.atlassian.net/browse/HTCONDOR-735
https://opensciencegrid.atlassian.net/browse/HTCONDOR-991

I'm using condor-9.0.17-1.el7.x86_64 and htcondor-ce-5.1.5-1.el7.noarch, the latest ones from the HTCSS production repo.

After some fiddling I was able to get it working creating token directories owned by the user,
storing there the token and copying only the token of the user submitting the job

JOB_ROUTER_CREATE_IDTOKEN_NAMES = fermilab2
JOB_ROUTER_IDTOKEN_REFRESH = 200

JOB_ROUTER_CREATE_IDTOKEN_fermilab2 @=end
    sub = "fermilabpilot@xxxxxxxx"
    kid = "POOL"
    lifetime = 900
    scope = "ADVERTISE_STARTD, ADVERTISE_MASTER, READ"
    # this dir is owned by "fermilab" otherwise it fails
    dir = "/etc/condor-ce/gltokens/fermilab"
    filename = "ce_fermilab2.idtoken"
    owner = "fermilab"
@end

JOB_ROUTER_SEND_ROUTE_IDTOKENS = fermilab2

This works for one user.
I want all jobs running on the CE to advertise to the monitoring collector.
If I add multiple tokens to JOB_ROUTER_SEND_ROUTE_IDTOKENS the jobs go on hold because of failed staging.
Any suggestion on the JOB_ROUTER_ROUTE_whatever to use to identify the user and send its token?

Even better, is it there a way to have a token sent to all users (different ownership once in the job directory)?
Or to generate automatically one token per user (without having to redefine manually the token for each user) and have it sent? 

Thank you,
Marco
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/