Re: [HTCondor-users] Docker cmdnotfound

[Michael Fienen <mike@xxxxxxxxxxx>]

Ahh…I think that’s it. I worked with Greg and confirmed that things are running. But…my script which calls the little python script is not running the version of python I installed inside the docker image. Unfortunately, it was installed in root inside docker since that’s the default user.

I think I just need to set the PYTHONPATH but I will also install not for root.

Thanks

Mike

On Aug 2, 2016, at 12:12 PM, Todd Tannenbaum <tannenba@xxxxxxxxxxx> wrote:

On 8/2/2016 7:22 AM, Michael Fienen wrote:

Hi Iain

Well, that’s embarrassing! That’s right, so fixed the typo. Now, if I
have executable = ./foo.sh, it runs but appears to run outside of docker
container (e.g. my script foo.py failed in such a way that it would not
if inside the container).

Are you sure it is running outside the container?  I.e. perhaps it is just not finding the proper files in your container due to missing environment variables such as PYTHONPATH or whatever.

I'd changing your submit file below to have something like
  executable = /bin/ls
  arguments = -l /etc
just to look around and confirm if you are really inside the docker filesystem or not...

Hope this helps
Todd

I tried “executable=foo.sh” without “./“ and

get the same error as before.

Getting closer though!

Thanks
Mike

On Aug 2, 2016, at 7:15 AM, Iain Bradford Steers <iain.steers@xxxxxxx
<mailto:iain.steers@xxxxxxx>> wrote:

Hi Mike,

I see you're transferring foo.py twice, but calling ./foo.sh?

Is that a typo or is foo.sh embedded in the docker image?

Cheers, Iain
________________________________________
From: HTCondor-users [htcondor-users-bounces@xxxxxxxxxxx
<mailto:htcondor-users-bounces@xxxxxxxxxxx>] on behalf of Michael
Fienen [mike@xxxxxxxxxxx <mailto:mike@xxxxxxxxxxx>]
Sent: 02 August 2016 14:10
To: HTCondor-Users Mail List
Subject: [HTCondor-users] Docker cmdnotfound

Hello HTCondorians!

I’m trying a trivial Docker universe job and it’s *almost* working.

I’m quite certain that HTCondor was able to make a running container
(it took forever the first time, presumably because pulling from
docker hub, and subsequent runs are faster but now hanging up).

Here’s my simple submit file:

universe=docker
docker_image=mnfienen/centos6_py35
notification=never
executable=./foo.sh
should_transfer_files=YES
output=out_$(Process).out
error=err_$(Process).err
log = log_$(Process).log
transfer_input_files = foo.py, foo.py
queue

And here’s what I get in the log before it goes to held status:
007 (234.000.000) 08/02 07:00:07 Shadow exception!
Error fromslot1_1@xxxxxxx
<mailto:slot1_1@xxxxxxx><mailto:slot1_1@xxxxxxx>.xxx.gov
<http://xxx.gov/><http://xxx.gov <http://xxx.gov/>>: Error running
docker job: cmdnotfound
0  -  Run Bytes Sent By Job
320  -  Run Bytes Received By Job
...
012 (234.000.000) 08/02 07:00:07 Job was held.
Error fromslot1_1@xxxxxxx
<mailto:slot1_1@xxxxxxx><mailto:slot1_1@xxxxxxx>.xxx.gov
<http://xxx.gov/><http://xxx.gov <http://xxx.gov/>>: Error running
docker job: cmdnotfound
Code 6 Subcode 0

I’ve tried a couple things for the executable argument (foo.sh is
simply running foo.py which works in the container locally). I tried
"executable=foo.sh”, “executable=./foo.sh”, and before that also tried
“executable=python foo.py”

I’m clearly doing something wrong.

Thanks for any help!

Cheers,
MIke Fienen
USGS Wisconsin Water Science Center



_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message tohtcondor-users-request@xxxxxxxxxxx
<mailto:htcondor-users-request@xxxxxxxxxxx>with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

-- 
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                  Madison, WI 53706-1685
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/