[Condor-users] Dagman/submit is slow

["Cargnelli, Matthieu" <Matthieu.Cargnelli@xxxxxxxx>]

Hi,

I'm trying to use condor to distrbute jobs within a cluster. I'm using a 
dagman to describe the 1200 tasks I wish to execute in parallel. It 
takes about 10 minutes to have a hundred of these tasks running. I have 
set the number of jobs the dagman can submit per interval to its maximum 
: 1000, and the interval to 0. But it submits only one or two tasks each 
second. It's sufficient to feed the procs I use for now, but I plan on 
using hundreds of processors soon, so I'd like to know if there's a way 
to speed things up.
The datafile I'm dealing with is approximately 2MB, and the network is 
one GB. From my experience in transmitting files over that network it 
mustn't take more than 0.2 s to be passed between the master and the nodes.
The nodes are configured to accept any work, never to preempt their jobs 
and are dedicated, do they are never interrupted by keypress or mouse. 
The universe I use is vanilla.

Thanks in advance for your help.

Matthieu Cargnelli