Re: [HTCondor-users] output file transfer using oauth token

[Dave Dykstra <dwd@xxxxxxxx>]

I agree the .top file or possibly the .meta stored in the credmon should
contain any additional information needed, instead of it being in .use.
Jobs should be able to set $BEARER_TOKEN_FILE to point to the .use file,
which will be understood by tools that follow the WLCG bearer token
discovery standard.

Dave

On Thu, Oct 27, 2022 at 09:10:03PM -0500, David Schultz wrote:
> Hi all,
> 
> Since I've been one of the rare users of the oauth2 credmon (and found my
> share of bugs), I'll add my viewpoint.  While I believe raw is a better
> format for the .use file, as the access token should be opaque to condor,
> it appears based on the oauth2 credmon and the various related plugins that
> the .use file was modeled off of the return value of the access token
> response in an oauth2 code flow:
> https://www.rfc-editor.org/rfc/rfc6749#section-4.1.4 
> 
> I believe there is only one place in the oauth2 credmon that cares about
> the json contents of the .use file:
> https://github.com/htcondor/htcondor/blob/main/src/condor_credd/condor_credmon_oauth/credmon/CredentialMonitors/OAuthCredmon.py#L64 
> 
> Here it looks at the expiration time of the access token to determine if it
> should renew using the refresh token.  That seems like a useful thing, but
> it's a field that could easily be written into the .meta or .top files if
> someone wanted to refactor the oauth2 credmon to use a raw access token.
> 
> However, all the other oauth plugins currently assume a json format as
> well, so those would also have to be updated.  I'm not sure if there's any
> appetite for such a breaking change though, even if there are very few
> users.
> 
> Best,
> David
> 
> 
> On Thu, Oct 27, 2022 at 4:34 PM Kevin Retzke via HTCondor-users <
> htcondor-users@xxxxxxxxxxx> wrote:
> 
> > Hi Todd,
> > Thanks for the fast response. We're using the Vault credmon. If this
> > isn't something affecting others we could look at writing our own file
> > transfer plugin (and very well may have to anyways when all is said and
> > done).
> >
> > Thanks,
> > Kevin
> > ------------------------------
> > *From:* Todd Tannenbaum <tannenba@xxxxxxxxxxx>
> > *Sent:* Thursday, October 27, 2022 5:00 PM
> > *To:* Kevin Retzke via HTCondor-users <htcondor-users@xxxxxxxxxxx>
> > *Cc:* Dave Dykstra <dwd@xxxxxxxx>; Kevin Retzke <kretzke@xxxxxxxx>
> > *Subject:* Re: [HTCondor-users] output file transfer using oauth token
> >
> > On 10/27/2022 3:14 PM, Kevin Retzke via HTCondor-users wrote:
> >
> > I'm trying to have a job upload output to dcache over https,
> > authenticating with a scitoken. In my submit file I have:
> >
> > use_oauth_services = dune
> > output_destination = dune+https://example.com/foo/bar
> >
> > Condor does send the token at $_CONDOR_CREDS/dune.use, but this file only
> > contains the encoded token itself, which we expect in order to use the
> > token in the job. However, uploading output fails with "Unable to parse
> > token as JSON", since the file transfer plugin expects the dune.use file to
> > be JSON with the token under the "access_token" field.
> > https://github.com/htcondor/htcondor/blob/main/src/condor_filetransfer_plugins/multifile_curl_plugin.cpp#L131 
> > <https://github.com/htcondor/htcondor/blob/main/src/condor_filetransfer_plugins/multifile_curl_plugin.cpp#L131>
> >
> > Is this a bug in the file transfer plugin, or can I just not use the oauth
> > token like that? The documentation suggests it should work in
> > https://htcondor.readthedocs.io/en/latest/users-manual/file-transfer.html#file-transfer-using-a-url 
> > <https://htcondor.readthedocs.io/en/latest/users-manual/file-transfer.html#file-transfer-using-a-url>
> >
> > condor 9.0.16 on the schedd
> >
> > Thanks,
> > Kevin
> >
> >
> > Hi Kevin,
> >
> > Thank you for the detailed report and your nice (source code)
> > investigative work above!
> >
> > Question: what issued you the scitoken?  was it Vault?  a local issuer?
> > or?
> >
> > Initial thinking here the .use file, if created by the oauth2 credmon, is
> > JSON.   But if it is the Vault credmon instead, then... looks like it is
> > raw.    Just want to confirm what you are using before we decide how to go
> > about fixing it.
> >
> > Thanks
> > Todd
> >
> > _______________________________________________
> > HTCondor-users mailing list
> > To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with
> > a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users 
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/htcondor-users/ 
> >

> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users 
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/