[HTCondor-users] Singularity: force mount specific directory (job scratch dir) fails when home dir is not accessible

[Thomas Hartmann <thomas.hartmann@xxxxxxx>]

Hi all,

I am trying to start all job's by a user in a Singularity container as
default.

The problem is now, that the user has it's nominal $HOME on a shared fs
- which is not supposed to be used here. So I would like to go with the
default
  SINGULARITY_TARGET_DIR = /srv
and mount the job's scratch dir under /srv and let $HOME point to it.

However, jobs fail as their user's nominal home directory is on the
shared fs and are naturally not accessible, as I am not forwarding any
token [1].
So, I wonder if there is a way to force overwrite $HOME of the user to
the scratch dir so that Condor can mount a directory actually owned by
the user as $HOME in the container [2]?
Or maybe if there is another way to forward something like the "--home"
flag to Singularity? (actually, just exporting a new HOME is not
considered by Singularity when mounting a path somewhere(?) into the
container [3]?)

Cheers,
  Thomas



[1]
ERROR  : Home directory is not owned by calling user:
/afs/desy.de/user/g/grid
ABORT  : Retval = 255


[2] on the WN as user 'grid'
> singularity shell --home /tmp/foo:/srv  /var/containers/sl6
Singularity: Invoking an interactive shell within container...

sh-4.1$ exit

[2.b] which is a bit cheated, as the user has its nominal $HOME not
accessible
> Unable to create and initialize directory '/afs/desy.de/user/g/grid'.
> echo $HOME
/afs/desy.de/user/g/grid

[3]
[grid@wn12-test]/tmp% export HOME=/tmp/foo
[grid@wn12-test]~% cd $HOME
[grid@wn12-test]~% pwd
/tmp/foo
[grid@wn12-test]~% singularity shell /var/containers/sl6
ERROR  : Home directory is not owned by calling user:
/afs/desy.de/user/g/grid
ABORT  : Retval = 255

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature