Re: [HTCondor-users] [EXTERNAL] Re: Accessing external drive with docker universe

["Fienen, Michael N" <mnfienen@xxxxxxxx>]

Actually – quick update. When I execute “ls” the mount directory doesn’t show up, but I added /mnt/condor_working/etc/blahblah/bin to the path to run a specific version of python and it worked! So…strange it doesn’t show up with ls but seems the mount worked.

 

 

 

--

Michael Fienen, Ph. D.
Research Hydrologist
United States Geological Survey

Upper Midwest Water Science Center
1 Gifford Pinchot Drive

Madison, Wisconsin 53726
phone:  608.821.3894
https://www.usgs.gov/staff-profiles/michael-n-fienen

 

 

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Fienen, Michael N via HTCondor-users <htcondor-users@xxxxxxxxxxx>
Date: Friday, July 22, 2022 at 1:18 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Cc: Fienen, Michael N <mnfienen@xxxxxxxx>
Subject: Re: [HTCondor-users] [EXTERNAL] Re: Accessing external drive with docker universe

Thanks Greg!

 

That does look like what I want. However, it seems I’m not executing it right. I added this to config.d/config on my worker machines:

# add /mnt/condor_working to be accessible to docker

DOCKER_VOLUMES = CONDOR_EFS

DOCKER_VOLUME_DIR_CONDOR_EFS = /mnt/condor_working

DOCKER_MOUNT_VOLUMES = CONDOR_EFS

 

But I can’t seem to find the drive when I run in docker universe. Should it be called the same path from docker in the condor job?

 

My executable is just running ls and I also tried to add a subdirectory to the PATH variable but isn’t finding things in there.

 

Thanks

mike

 

 

--

Michael Fienen, Ph. D.
Research Hydrologist
United States Geological Survey

Upper Midwest Water Science Center
1 Gifford Pinchot Drive

Madison, Wisconsin 53726
phone:  608.821.3894
https://www.usgs.gov/staff-profiles/michael-n-fienen

 

 

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Greg Thain <gthain@xxxxxxxxxxx>
Date: Friday, July 22, 2022 at 12:54 PM
To: htcondor-users@xxxxxxxxxxx <htcondor-users@xxxxxxxxxxx>
Subject: [EXTERNAL] Re: [HTCondor-users] Accessing external drive with docker universe

 

 This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.  

 

On 7/22/22 11:36, Fienen, Michael N via HTCondor-users wrote:

Hello

 

I am trying to run a docker universe job where I have some external data in a location like /mnt/working on each worker. This external data is read only, but I can’t seem to be able to set the path to it from within docker. It was looking like MOUNT_UNDER_SCRATCH would be the ticket but trying to use export MOUNT_UNDER_SCRATCH=’/mnt/working’ in my executable script fails. Is there a better way to do this?

 

Mike:

 

I think you want DOCKER_VOLUMES:

https://htcondor.readthedocs.io/en/latest/admin-manual/setting-up-vm-docker-universes.html#the-docker-universe

 

-greg

 

Many thanks in advance!

Mike

 

--

Michael Fienen, Ph. D.
Research Hydrologist
United States Geological Survey

Upper Midwest Water Science Center
1 Gifford Pinchot Drive

Madison, Wisconsin 53726
phone:  608.821.3894
https://www.usgs.gov/staff-profiles/michael-n-fienen

 

 

_______________________________________________

HTCondor-users mailing list

To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a

subject: Unsubscribe

You can also unsubscribe by visiting

https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:

https://lists.cs.wisc.edu/archive/htcondor-users/