Re: [Condor-devel] too many links in spool directory

[Matthew Farrellee <matt@xxxxxxxxxx>]

On 10/12/2010 12:47 PM, Dan Bradley wrote:
> Hi all,
>
> If you care about the spool directory, please take a glance at the
> following ticket which proposes adding a hierarchy of directories.
>
> I'd like to finish working on it this week (in time for the CHEP
> conference next week!), so that we can demonstrate that Condor scales to
> more than 32k running jobs.
>
> https://condor-wiki.cs.wisc.edu/index.cgi/tktview?tn=1706
>
> Thanks,
> --Dan

Not spitting hairs - "Condor scales to more than 32K running jobs" -> "a 
single Condor Schedd scales to moer than 32K running jobs"?

willb has done work to avoid spool creation for jobs that do not need 
them. I understand your workload may require the spool, but if not you 
may use his work to test >32K without code changes.

Any idea why we haven't run into this problem much in the past? It is 
not uncommon to have >32K spooled jobs.

Best,


matt