[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] d/l job logs from remote submit / schedd node with py bindings?



The job submission interface (whether python bindings or condor_submit command-line tool) assumes that all files and directories mentioned in your submit description are accessible where youâre submitting from. It also assumes one of the following is true:
1) The schedd has access to the files/directories (i.e. itâs running on the same machine).
2) You will use the spool option, in which case the submit operation includes reading the files locally and transfering them over the network to the schedd. After the job completes, you can retrieve the output files over the network from the schedd.

 - Jaime

> On Sep 30, 2025, at 1:57âPM, Gavin Price <gaprice@xxxxxxx> wrote:
> 
> Hi all,
> 
> Not sure I have the vocabulary right here, but I'm running condor jobs on a remote cluster using the python bindings. There are no condor daemons running on the local machine. I think the answer is no from looking at the python API, but is there any way to stream the job `log`, `output`, and `error` logs (e.g. the files created based on the fields in the Submit() class) from the remote schedd / submit node to the local node?
> 
> If not, a feature to add a method to fetch logfiles from a remote submit / schedd node using the python bindings would be helpful.
> 
> Thanks very much,
> 
> Gavin
> 
> P.S. it would also be nice if the requirement for the `initialdir` to exist on the local node could be toggled off, since AFAICT there's no need for it to exist when the condor daemons are all fully remote, but that's a minor issue