[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Condor-users] Scheduling N of M jobs
- Date: Fri, 05 Feb 2010 10:04:03 -0600
- From: Nathan Whitehorn <nwhitehorn@xxxxxxxxxxxxxxxx>
- Subject: [Condor-users] Scheduling N of M jobs
I am working with a set of jobs that are I/O bound, and need to submit
several hundred of them to our cluster. Once 10 or 20 are running, the
storage system they are using becomes saturated, so running more than
that just wastes CPUs in the cluster. Is there a way, as a normal user,
to tell the scheduler that, of this group of 400 jobs, I want 10 to be
running at any given time?
This seems to be a fairly common problem, but all I have run into as a
solution are scripts that periodically check the current queue state and
either submit or release jobs if the running number has dropped below a
threshold, which seems like an ugly hack.
-Nathan