[Condor-users] Request for Ideas/Plans: Designing a Large Condor Pool

[Jess Cannata <jac67@xxxxxxxxxxxxxx>]

Dear Condor User Community:

We are in the process of setting up a Condor pool to initially include 
all lab machines (400-1000 machines) on campus, though later we plan to 
add a few of our clusters. While we currently run Condor on some of our 
smaller clusters, we suspect that the layout for this larger pool will 
be different than a standard Condor pool.

For this campus pool, we want one entry point for users to submit jobs. 
Since the pool will have tens of thousands of jobs in queue, with 
several hundreds running simultaneously, we know that we will likely 
overload one schedd along with the other daemons.

Does anyone have any design plans that outline how one might set up a 
pool with a single point of entry, with multiple daemons to spread out 
the load and provide some redundancy? I've looked in the manual for 
examples of large deployments, but cannot find any. Am I missing 
something? If you wouldn't mind sharing your pool layout, I think that 
this would be useful to many Condor users especially if your pool is not 
a typical pool.

--
Jess Cannata
Advanced Research Computing
Georgetown University