Re: [HTCondor-users] output file transfer using oauth token

[David Schultz <david.schultz@xxxxxxxxxxxxxxxx>]

Hi all,

Since I've been one of the rare users of the oauth2 credmon (and found my share of bugs), I'll add my viewpoint. While I believe raw is a better format for the .use file, as the access token should be opaque to condor, it appears based on the oauth2 credmon and the various related plugins that the .use file was modeled off of the return value of the access token response in an oauth2 code flow:

https://www.rfc-editor.org/rfc/rfc6749#section-4.1.4

I believe there is only one place in the oauth2 credmon that cares about the json contents of the .use file:

https://github.com/htcondor/htcondor/blob/main/src/condor_credd/condor_credmon_oauth/credmon/CredentialMonitors/OAuthCredmon.py#L64

Here it looks at the expiration time of the access token to determine if it should renew using the refresh token. That seems like a useful thing, but it's a field that could easily be written into the .meta or .top files if someone wanted to refactor the oauth2 credmon to use a raw access token.

However, all the other oauth plugins currently assume a json format as well, so those would also have to be updated. I'm not sure if there's any appetite for such a breaking change though, even if there are very few users.

Best,

David

On Thu, Oct 27, 2022 at 4:34 PM Kevin Retzke via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:

Hi Todd,

Thanks for the fast response. We're using the Vault credmon. If this isn't something affecting others we could look at writing our own file transfer plugin (and very well may have to anyways when all is said and done).

Thanks,

Kevin

---

From: Todd Tannenbaum <tannenba@xxxxxxxxxxx>
Sent: Thursday, October 27, 2022 5:00 PM
To: Kevin Retzke via HTCondor-users <htcondor-users@xxxxxxxxxxx>
Cc: Dave Dykstra <dwd@xxxxxxxx>; Kevin Retzke <kretzke@xxxxxxxx>
Subject: Re: [HTCondor-users] output file transfer using oauth token

Â

On 10/27/2022 3:14 PM, Kevin Retzke via HTCondor-users wrote:

I'm trying to have a job upload output to dcache over https, authenticating with a scitoken. In my submit file I have:

use_oauth_services = dune
output_destination = dune+https://example.com/foo/bar

Condor does send the token at $_CONDOR_CREDS/dune.use, but this file only contains the encoded token itself, which we expect in order to use the token in the job. However, uploading output fails with "Unable to parse token as JSON", since the file transfer plugin expects the dune.use file to be JSON with the token under the "access_token" field. https://github.com/htcondor/htcondor/blob/main/src/condor_filetransfer_plugins/multifile_curl_plugin.cpp#L131

Is this a bug in the file transfer plugin, or can I just not use the oauth token like that? The documentation suggests it should work in https://htcondor.readthedocs.io/en/latest/users-manual/file-transfer.html#file-transfer-using-a-url

condor 9.0.16 on the schedd

Thanks,

Kevin

Hi Kevin,

Thank you for the detailed report and your nice (source code) investigative work above!

Question: what issued you the scitoken? was it Vault? a local issuer? or?

Initial thinking here the .use file, if created by the oauth2 credmon, is JSON.ÂÂ But if it is the Vault credmon instead, then... looks like it is raw.ÂÂÂ Just want to confirm what you are using before we decide how to go about fixing it.

Thanks
Todd

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/