[HTCondor-users] File transfer not working with docker universe ?

[Gaetan Geffroy <gage@xxxxxxxxx>]

Hi,

 

I am submitting the following job (through python):

import htcondor

s = htcondor.Schedd()

 

job = htcondor.Submit({

    "executable": "/bin/cat",

    "arguments": "test.txt",

    "transfer_executable": False,

    "initialdir": "/tmp",

    "should_transfer_files": True,

    "transfer_input_files": "/test/test.txt",

    "universe": "docker",

    "docker_image": "python:3.8.10",

    "output": "/tmp/test.out",

    "error": "/tmp/test.err",

    "max_retries": 0,

})

 

with s.transaction() as t:

    job.queue(t)

Once it ran, test.out is empty and test.err contains the following: /bin/cat: test.py: No such file or directory

 

However, if I change my job for a Vanilla universe one:

import htcondor

s = htcondor.Schedd()

 

job = htcondor.Submit({

    "executable": "/bin/cat",

    "arguments": "test.txt",

    "transfer_executable": False,

    "initialdir": "/tmp",

    "should_transfer_files": True,

    "transfer_input_files": "/test/test.txt",

    "universe": "vanila",

    "output": "/tmp/test.out",

    "error": "/tmp/test.err",

    "max_retries": 0,

})

 

with s.transaction() as t:

    job.queue(t)

Then it works as intended. This is a dummy recreation of a problem I am having in a bigger project, and which shows the same behavior: input files not being transferred in the docker container.

 

Thanks,

Gaëtan

 

Gaetan Geffroy
Junior Software Engineer, Space

Terma GmbH
Europaarkaden II, Bratustraße 7, 64293 Darmstadt, Germany
T +49 6151 86005 43 (direct)  •  T +49 6151 86005-0
Terma GmbH - Sitz Darmstadt  •  Handelsregister Nr.: HRB 7411, Darmstadt
Geschäftsführer: Poul Vigh / Steen Vejby Sørensen
www.terma.com • Linkedin • Twitter • Instagram • Youtube

---

Attention:
This e-mail (and attachment(s), if any) - intended for the addressee(s) only - may contain confidential, copyright, or legally privileged information or material, and no one else is authorized to read, print, store, copy, forward, or otherwise use or disclose any part of its contents or attachment(s) in any form. If you have received this e-mail in error, please notify me by telephone or return e-mail, and delete this e-mail and attachment(s). Thank you.