Re: [HTCondor-users] S3 Protocol Usage for transfer files

[Ram Ban <ramban046@xxxxxxxxx>]

Hello Todd,Â

Thanks for your response.

I thought files would be directly transferred from Execute to s3 and thus would reduce load on AP(submitter).

But if it would come to AP and then go to s3, then it would increase the load on AP ?

ThanksÂ

Raman

On Fri, Mar 13, 2026, 02:53 Todd L Miller via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:

> I want to know how much it is reliable?

    As far as I know, we've never seen problems with our client
implementation. I can't speak to how reliable using any particular S3
service might be.

    Internally, HTCondor converts S3 URLs into HTTPS URLs by
"pre-signing" them on the AP. (This is done so that your S3 credentials
never have to leave the AP.)Â Thus, the error handling for S3 is generally
going to be identical to using HTTPS URLs directly.

-- ToddM
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe

The archives can be found at: https://www-auth.cs.wisc.edu/lists/htcondor-users/