Re: [HTCondor-users] Support for ambient credentials in built-in S3 plugin

[Miguel Garrido <miguel@xxxxxxxxx>]

Per the documentation there is native S3 support, perhaps the curl plugin is used for this?

https://htcondor.readthedocs.io/en/latest/users-manual/file-transfer.html#transferring-files-using-the-s3-protocol

I didnât save the failed attempt logs, though the only failure I saw was trying to submit a cluster without setting an s3 private key for the S3 url signing.

Thanks

On Mon, Mar 6, 2023 at 15:45 Todd L Miller via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:

> Hello, I am using Condor 9.0.16 on Windows and I am trying to use the
> native S3 file transfer capabilities with an EC2 instance's ambient
> credentials (via its instance profile) to write to an S3 bucket.

    Right now, the only HTCondor tool that supports using credentials
from the instance profile is condor_annex.

> According to the Condor logs everything worked, but the files
> are not in S3.

    This seems wrong. IIRC, HTCondor doesn't ship with an
S3-supporting plug-in, so file transfer should have failed when it
couldn't find the corresponding plug-in. Do you have logs of a failed
attempt? (If the log you sent is of a failed attempt, could you set
STARTER_DEBUG = D_FULLDEBUG on the startd, and send the StarterLog?
Thanks.)

- ToddM
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

--

MG