Re: [HTCondor-users] equivalent job array option for condor_submit

["Upton, Stephen (Steve) (CIV)" <scupton@xxxxxxx>]

Thanks, Todd and Michael! I guess I should have RTM a bit more carefully.

 

steve

 

Stephen C. Upton

SEED (Simulation Experiments & Efficient Designs) Center

Mobile: 804-994-4257

 

From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Michael V Pelletier
Sent: Thursday, August 04, 2016 5:52 PM
To: HTCondor-Users Mail List
Subject: Re: [HTCondor-users] equivalent job array option for condor_submit

 

From: "Upton, Stephen (Steve) (CIV)" <scupton@xxxxxxx>
Date: 08/04/2016 04:36 PM

> Is there an equivalent to job array like in SLURM or PBS where you can
> submit specific job numbers? Iâm aware of queue, but that seems to queue
> up a specified number of  jobs, starting from 0. Iâm looking for the
> ability to specify a set of job numbers or ranges, that kind of thing. I
> didnât find anything googling nor in the docs (but could be looking in the
> wrong place or using a different terminology).

The trick here is to do math with the $(Process) macro in the submit
description, or use the 8.4 "queue in" feature.

Prior to 8.4 this was a bit peculiar since you had to use the $$()
format with the ProcID attribute or a ClassAd array, instead of the
submit description macros, but now it's quite a bit easier.

So to do something numbered from 1 instead of zero:

CASE_ID = $(Process) + 1
arguments = --case=$INT(CASE_ID)

The argument will become "--case=1" for process 0, and so on.

To do something numbered 10, 20, 30, etc, it's the same idea:

CASE_ID = $(Process) * 10 + 10

The "queue in," "queue matching," and "queue from" commands
allow you to specify Python-style slices as start:end:step,
so you can constrain the runs that way.

For example, if you want to run the first three values from
your list of RNG seeds:

queue SEED in 0:3 ( 123, 456, 789, 012, 345, 678, 901)

This would set $(SEED) to 123, 456, and 789 in processes
0, 1, and 2 respectively.

        -Michael Pelletier.
_

Attachment: smime.p7s
Description: S/MIME cryptographic signature