Mailing List Archives
Authenticated access
|
|
|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [HTCondor-users] Singularity not mounting the home directory for a submitted job
- Date: Wed, 17 Oct 2018 19:50:25 +0000
- From: Tom Downes <downes@xxxxxxx>
- Subject: Re: [HTCondor-users] Singularity not mounting the home directory for a submitted job
Michael:
What I'd wager is happening is that you have
mount home = yes
in your singularity.conf file. This has the effect of bind mounting $HOME onto /home/user-running-singularity in the container. Which means you can't bind mount host:/home onto the non-empty container:/home. If you have a situation where a job needs BOTH /home/A and /home/B, you should probably set "mount home = no" and then
bind path = /home
or the corresponding SINGULARITY_BIND_EXPR.
Tom
ïOn 10/15/18, 4:50 PM, "HTCondor-users on behalf of Todd Tannenbaum" <htcondor-users-bounces@xxxxxxxxxxx on behalf of tannenba@xxxxxxxxxxx> wrote:
On 10/15/2018 1:42 PM, Michael Pelletier wrote:
> Hi folks,
>
> Does anyone have some insight into how Singularity decides when to bind-mount the user's home directory?
>
> I've got a situation where when I run the command on the exec machine directly, the container spins up successfully with the home directory in place, but when started via condor_run -a 'MY.SingularityImage = "imagefile"' or via a submit description, the container doesn't get the home directory and thus the launch of the application fails.
>
> The bind mount arguments include the /home/condor directory since that's where the global libexec lives in my config, but omitting that didn't change the behavior. An error message I see is "/home/user already mounted in container" - so it's apparently detecting something that isn't actually there.
>
> I have a user_job_wrapper in place, but omitting that didn't change this behavior. I wrapped the Singularity binary as well in order to add the "--nv" option for jobs where it's appropriate until SINGULARITY_EXTRA_ARGUMENTS is in a production release, but that was another dead end investigation. I tried inserting a "-H ${HOME}:$HOME" argument along with --nv, but that seems to be getting ignored with the "already mounted in container" message.
>
> What's even more peculiar is that it works for some user accounts, but not others. I checked the environment variables for any differences, but nothing jumped out at me, and condor_run does a getenv=True in any case.
>
> Any suggestions? Thanks
>
I am not the Singularity support expert, but maybe HTCondor config knob
SINGULARITY_BIND_EXPR could help? Ie something like:
SINGULARITY_BIND_EXPR = strcat("/home/",Owner)
regards,
Todd
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/