Re: [HTCondor-users] Passing IDToken to user job for identification to storage resource

["Bockelman, Brian" <BBockelman@xxxxxxxxxxxxx>]

Hi Benoit,

You might be interested in the OAuth2 credmon component for HTCondor.  This helps manage the token and push it to the remote side.

There are two implementations shipped:

- The vault credmon.  This is maintained by Dave Dykstra.  The Vault server helps avoid needing to do OAuth2 workflows out to the end-client (which can be awkward).

- The OAuth2 credmon.  This includes a small webserver that does the web-based OAuth2 flow.

If you want to continue using oidc-agent, perhaps it would be useful to contribute an oidc-agent credmon?  It's python code, a fairly approachable part of the codebase, and I imagine it would benefit many others.

Once HTCondor manages the token, the user can even utilize the built-in HTTP file transfer plugin and simply put the URL in transfer_output_files / transfer_output_remaps.

Hope this helps,

Brian

On Dec 2, 2022, at 6:37 AM, Benoit Roland <benoit.roland@xxxxxxx> wrote:

Dear all,

our jobs need to store their output on an storage resource (no shared file system) which requires IDToken authentication.

The IDToken is retrieved from the provider by the user on the command line before the job submission.

I read about different options to manage credentials, but I am not exactly sure about which one is more appropriate for our use case.

The flow is as follows:

1. Before submission, the token is generated on the command line: oidc-token -f punch-aai >> idtoken.text
 
2. We would like to store this token in a secured place defined in the HTCondor configuration to make it accessible to the job submitted by the user.
    So far, for test purpose, the token is part of the "transfer_input_files" in the submit description file.

3. The transfer of the job output to the storage resource is so far hardcoded in a python script (included as well in the jdl "transfer_input_files") doing essentially:

  "curl -L -X PUT -H \"Authorization: Bearer " + str(TOKEN) + "\"" + " --upload-file " + INPUTFILE + " " + STORAGE_RESOURCE

4. Not yet implemented is the refreshment or renewal of the IDToken which is limited to a one hour lifetime.

We would like to ask you which approach/option would be more suited to our use case?

Thanks a lot in advance for your answer and advice!

Cheers,
Benoit

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/