[Condor-users] Support for user-driven tiered jobs?

[Ian Stokes-Rees <ijstokes@xxxxxxxxxxxxxxxxxxx>]

We are faced with a problem where jobs from one user within a group
block out all jobs by another user within the same user group.  We would
like to have a tiered system that is *user* driven to specify "Within my
user group, place this job on Tier N", so that a large set of jobs by
one user (i.e. possibly several weeks of full cluster usage) could be
set at a lower priority, thereby allowing jobs from another user within
the same group to take priority.  Sort of putting the jobs into a
"backfill" mode within the user group, but NOT "overall backfill" --
they should compete equally with all jobs from other user groups.

Then, when a machine classad is being matched, and user A and user B,
both within the same user group, have two jobs X and Y respectively
that  in all other ways are identical, the match will prefer the job
with the higher (better) tier.

Right now we can't see how that is possible from the user side -- users
can't put in their Ranking expression anything that refers to other jobs
that are being considered at the same time.  A job can only consider the
machine it is being matched against.  It is the machine that has the
power to decide between jobs.

I realize there are some simple things that can be put into the machine
Rank expression, however my situation is complicated by the fact that
this is happening in a grid environment, so:

* The names of the user groups is large and not easily listed/known a
priori.

* The implementation of this policy would  need to be done by many
independent condor deployments, so would need to have minimal or no
effect on jobs that do not want to use this system.

* The implementation of this policy would need to be (relatively) simple
in order to get agreement.

* Ideally some clever user-based solution would be possible that would
not require changing machine classads.

For completeness, we are aware that glide-in frameworks often provide
this kind of functionality, however we're trying to see what we can
achieve without considering glide-in (yet).

Ian

-- 
Ian Stokes-Rees, PhD                       W: http://hkl.hms.harvard.edu
ijstokes@xxxxxxxxxxxxxxxxxxx               T: +1 617 432-5608 x75
NEBioGrid, Harvard Medical School          C: +1 617 331-5993


begin:vcard
fn:Ian Stokes-Rees, PhD
n:Stokes-Rees;Ian
org:Harvard Medical School;Biological Chemistry and Molecular Pharmacology
adr;dom:;;250 Longwood Ave;Boston;MA;02115
email;internet:ijstokes@xxxxxxxxxxxxxxxxxxx
title:Research Associate, Sliz Lab
tel;work:+1 617 432-5608 x75
tel;fax:+1 617 432-5600
tel;cell:+1 617 331-5993
url:http://hkl.hms.harvard.edu
version:2.1
end:vcard