Re: [HTCondor-users] Support for ambient credentials in built-in S3 plugin

[Tom Downes <tpdownes@xxxxxxxxx>]

Miguel-

I think you're trying to save a bit of code, but I thinkÂdirectly reading the files over HTTPS (with headers) is the most future-proof solution.

For example, I think - but am not certain - that, on AWS, the IAM permission to read an object is sufficient to generate the presigned URL to GET the object over HTTPS. I think that may also be true for writing objects and PUT to a signedÂURL. So it would *probably* work on AWS but I obviouslyÂcannot and do not speak with authority.

That said, the solution would probably not work for Google Cloud Storage because HTCondor is relying upon a "S3 interoperability" mode with HMAC authentication. It would have to do some extra steps to get the HMAC key/secret and use the "XML API".

By contrast, if your Instance Profile or GCP Service Account has the permission to read/write the object, you know with 100% confidence that you can GET/PUT the documentedÂURLs and without managing URL expiration.

Tom

On Sat, Mar 11, 2023 at 12:12âPM Miguel Garrido <miguel@xxxxxxxxx> wrote:

Iâd like to add that the approach Iâve outlined in my previous message would actually work in tandem with SIGN_S3_URLS = True because the startd would be able to get the credentials to sign the url for the S3 operation (instead of the existing behavior which apparently does so on the submit machine).

On Sat, Mar 11, 2023 at 13:09 Miguel Garrido <miguel@xxxxxxxxx> wrote:

Hi Tom

This is indeed another approach which could yield a very elegant SDK-less solution to this problem. Here is the relevant AWS documentation for retrieving the temporary credentials via cURL:Â

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html#instance-metadata-security-credentials

If Condor were to support a role name parameter at submission instead of a key and then it followed the steps above on the startd to obtain temporary credentials at file transfer time (not submit time), I think this approach could work.

Thanks!

On Sat, Mar 11, 2023 at 12:52 Tom Downes <tpdownes@xxxxxxxxx> wrote:

Miguel:

I maintain the HTCondor solution for Google Cloud and face a similar challenge. The basic issue is that it's not typically considered aÂgood practice to store credentials in files, especially when they probably have no expiration or a very lengthy expiration. The Condor teamÂfaces a real challenge in supporting a variety of use cases but I think this is a missingÂgap.

What would be interesting to me is if the behavior of S3_SIGN_URLs = False were to make the assumption that the execute points have credentials available to them. In that case, it would implement something like the following on the execute hosts:

GCP:

curl -X GET -o file1 -H "Authorization: Bearer $(gcloud auth print-access-token)" "https://storage.googleapis.com/storage/v1/b/BUCKET/o/OBJECT?alt=media"

AWS (pattern documented here):

curl -X GET -o file1 -H "(CORRECT AWS HEADERS)"Â https://BUCKET.s3.REGION.amazonaws.com/OBJECT

I believe AWS buckets expose their region to unauthenticated users, but that should be subjected to tests.

This could be written in a way where the authorization tokens did not depend upon the presence of gcloud/aws clients, but queried the local instance metadata servers directly for an auth token/header.

In any case, this is all HTTPS and potentially all able to be done with cURL or equivalent.

Tom

On Mon, Mar 6, 2023 at 4:16âPM Miguel Garrido <miguel@xxxxxxxxx> wrote:

Thank you, I noticed that sentence as well. However, I also noticed that if I turned off pre signing then Condor would report that it is transferring files, hence my initial message to this list.

If using instance profile credentials is not something that Condor plans to support perhaps the correct behavior is for Condor to âignoreâ the transfer if there isnât a pre signed URL - since thatâs the only âsupported wayâ out of the box.

Iâll continue supporting my file transfer plugin for now.Â

On Mon, Mar 6, 2023 at 17:10 Todd L Miller <tlmiller@xxxxxxxxxxx> wrote:

> Per the documentation there is native S3 support, perhaps the curl plugin
> is used for this?

    The native S3 support requires you presigning to happen; see the
third sentence in the first paragraph of the documentation to which you
linked.

- ToddM

--

MG

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

--

MG

--

MG