So we've got a situation where certain
jobs can run better if a GPU is available, but can still run fine without
one. This poses an interesting problem - the "request_gpus" line
bakes that request into the job requirements _expression_, so that the job
won't run unless it can get a GPU for itself.
Is there any clever trick anyone can
think of which will reserve a GPU and set up the appropriate environment
variables if one is available, but still allow the job to match and run
even if it can't? I think my difficulty is that I'm not quite clear on
how a request for a resource becomes a deduction from the pool of that
resource on the machine - I think it comes down to needing the deduction
to take place when it matches even if it's not mandatory for the job.