Re: [Condor-users] limit on input file transfers ????

["Dr Ian C. Smith" <i.c.smith@xxxxxxxxxxxxxxx>]

--On 30 March 2005 12:22 -0600 Zachary Miller <zmiller@xxxxxxxxxxx> wrote:

> > Have either of you tried this under 6.7?  The file transfer in 6.7
> > should  handle this.
>
> i'm sure it would, it's just a matter of efficiency.  sending over an
> entire BLAST db for one condor job does not yield very good througput.

No but if you can run a lot of searches in a single condor job
over say several hours then the startup time becomes less of an overhead.

> alain's use of rsync, combined with extracting only the useful data from
> the blast output is quite nice.  i plan on moving our BLAST framework
> here at the UW to something similar.

But there's no rysnc for windows AFAIK. There is also the problem of
permissions. The db files need to be writable for the updates to work
and thus need to put in the user writable area. This is deliberately 
volatile
so that junk doesn't accumulate in it. Also having relatively large amounts
of disk tied up with condor on a semi-permanent basis is unlikely to
be politically popular.

I'll try 6.7

thanks,

-ian.

> cheers,
> -zach
>
> _______________________________________________
> Condor-users mailing list
> Condor-users@xxxxxxxxxxx
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users