Re: [HTCondor-users] How to limit number of machines used in a long list of queues

[<Greg.Hitchen@xxxxxxxx>]

MAX_JOBS_RUNNING = 50

 

in the config of your submit node should do the trick.

 

You should also be able to use $(Process) in your submit file to simplify it.

 

e.g.

 

initialdir = dir_$(Process)

queue 1000

 

You may need to do something fancier as the above will produce

dir_0

dir_1

.....

dir_10

.....

dir_9999

 

initialdir = dir_$$([$(Process)+1])

should give you 1-1000 rather than 0-9999

but still needs some formatting to produce 4 digit numbers.

 

Cheers

 

Greg

 

From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Ralph Finch
Sent: Wednesday, 3 September 2014 6:34 AM
To: htcondor-users@xxxxxxxxxxx
Subject: [HTCondor-users] How to limit number of machines used in a long list of queues

 

HTCondor 8.2.1 on all Windowsx64 pool

I have about 80 machines (slots) in the pool. I need to submit more than 1000 jobs in a single submit file as follows:

initialdir = dir_0001

queue

initialdir = dir_0002

queue
....

initialdir = dir_1000

queue

OK, that will work, but I would like to limit the number of machines claimed at any time to, say, 50. How can I submit 1000 jobs, but have the number of machines actually used (50) limited to less than the number of machines available (80) at all times?

I don't want to have to explicitly list each machine allowed or disallowed using a Requirements statement.  Instead I'd like to say something like

Max_Machines = 50

 

Thanks much--

Ralph Finch

Calif Dept. of Water Resources
Sacramento, CA USA