Re: [Condor-users] Transfer of output files very slow

["user790 user790" <user790@xxxxxxxxx>]

I am running the latest version, condor 7.2.0. Very slow means about 15 minutes for transfering 10MB. I'll indeed dig into security checks to see if I can improve this latency, thanks.

 

P.

2008/12/31 Matthew Farrellee <matt@xxxxxxxxxx>

user790 user790 wrote:
> I am running Condor on two Windows XP computers. My program is an
> uninstrumented binary running in a vanilla universe. During its execution it
> writes a large (~300) number of small (~1 to 100 kB) files. Now after the
> program has been executed, the files that has been written locally on disk
> are transfered back to the submitter very slowly. It seems it is due to the
> number of files rather than the amount of data, because if I concatenate all
> the files into a single uncompressed archive, it is transferred much faster.
>
> Any idea what is going on and how I could avoid the extra compression step?
>
> Thanks,
>
> P.

On Windows, Condor will do a permissions/ACL check on both sides of the
file transfer, for each file. Maybe that is an expensive operation in
your environment.

(BTW, what version of Condor? What's "very slow"?)

Best,


matt
_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/condor-users/