Re: [HTCondor-users] Support for ambient credentials in built-in S3 plugin

[Miguel Garrido <miguel@xxxxxxxxx>]

I think thatâs a great idea. I will add it to my backlog but probably wonât allocate time to do this until we upgrade past 9.7.x because even though the old plug-in spec isnât deprecated, itâs worthwhile staying ahead of deprecation and leveraging multiple file transfer support.

Thanks

On Mon, Mar 13, 2023 at 17:34 Todd L Miller via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:

> I wrote a transfer plugin using the Python AWS SDK that has worked for us
> since the 8.8 series (before native S3 support was introduced) and Iâll
> continue using that until we upgrade past 9.7.x (when I think the new file
> transfer spec was introduced) and I have to update it to follow the new
> spec;

    I would like to deprecate the single-file (old) plug-in spec, but
I don't think we have yet.

> FWIW: I also think it's reasonable for this use case to simply assume
> the existence of the "aws" or "gcloud" command line utilities in the
> PATH and put a shell script into src/condor_scripts to bake this feature
> in. If the HTCondor team is open to that idea, I'd be willing to (in
> time) develop one.

    My suggestion was going to be to write two plug-ins doing exactly
this, but name them something other than "s3". Perhaps 'ambients3' and
'ambientgs'? The HTCondor team would be happy to accept such plug-ins as
contributions. Such a plug-in could even check, in its handling of
`-classad`, if it had been configured on a machine without the tooling
and/or the required metadata server.

- ToddM_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

--

MG