[HTCondor-devel] RFC: Submissions

[Matthew Farrellee <matt@xxxxxxxxxx>]

There's a need to aggregate jobs into logical groups for performing 
aggregate operations and gathering metrics.
Condor does not explicitly provide such a feature. Users can choose to 
attribute their jobs and perform aggregate operations via constraints.
Condor should provide a recommended path for such aggregation, while 
allowing custom user paths.
Historically, a cluster (defined by the ClusterId attribute) partially 
operated in this fashion. However, clusters are: 0) Closed - the set of 
jobs within a cluster cannot be extended; 1) Partitioned - a single user 
action, a condor_submit, may result in multiple clusters; 2) Local - 
clusters cannot span multiple submission points (condor_schedds) in a pool.
A cluster has the property of local uniqueness. A submission need not be 
locally or globally unique. Globally, to eliminate the need for a new 
architectural role, which could hinder architectural flexibility and 
scalability. Locally, to allow users flexibility in specifying the 
submission's form.
A submission may not be locally or globally unique and therefore 
requires scoping within an entity that is globally unique. Naturally, a 
submission should be scoped within a user.
Open questions -
 . What definition of user for scoping? Owner, AccountingGroup, User?
 . What form should a Submission take? schema, free-form, case?

Plan -
 . Add submission command to condor_submit, tied to ATTR_JOB_SUBMISSION
 . Add default generation for ATTR_JOB_SUBMISSION to condor_schedd
 . Add submission as a unit to operate on to q, rm, hold, release, 
remove, suspend, continue
 . Optimize handling of submissions, e.g. index in condor_schedd