Re: [HTCondor-users] empty output directory locally retrieved while performing transfer to remote storage

[Benoit Roland <benoit.roland@xxxxxxx>]

Dear Greg,

thanks a lot for your answer, and sorry for my late reply.

Your description is correct.

On the remote storage, the directories and subdirectories are transferred successfully with their file content.

On the submit node, the directories and subdirectories are transferred back, without any file content.

And using the remap, the issue disappears.

Cheers,
ben

On 13/03/2024 23:21, Greg Thain via HTCondor-users wrote:
> On 3/10/24 12:51 PM, Benoit Roland wrote:
> > The output directory "results" and its subdirectories are transferred to the submit node as well, with an empty content.
>
>
> Do you mean that the directory tree with subdirectories is transferred back, but all the non-directory files are missing? This is surprising.
>
> > Using a configuration with a remap:
> >
> > use_oauth_services = helmholtz
> > should_transfer_files = YES
> > transfer_output_files = results
> > transfer_output_remaps = "results = helmholtz+https://dcache-desy-webdav.desy.de:2880//pnfs/desy.de/punch/benoit_roland/test_alexander/results";
> >
> > solves the issue, namely the output directory "results" and its subdirectories are successfully transferred to the remote storage, and nothing to the submit node.
>
>
> The remap is what we intended to work, but the other way shouldn't be so buggy. Let us look into fixing this.
>
>
> -greg
>
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/