[HTCondor-users] File transfer plugins & stdout/err

[Ben Jones <ben.dylan.jones@xxxxxxx>]

Hi,

I noticed whilst writing a plugin that naming for stdout/err seems to be ignored when using âouput_destinationâ, and confirmed the behaviour with the builtin http plugin. Is there a way to get stdout/err transferred without it keeping the sandbox _condor_stdout/_condor_stderr names? Iâve tried with various transfer_output_remaps, but a solution evades me. This is with 9.0.9 on the exec node. 

Example:


Submit file:

$ cat testhttp.sub
executable     = yf.sh
arguments      = $(ClusterId) $(ProcId)
log            = xfer.$(ClusterId).log
error          = yf.$(ClusterId).$(ProcId).err
output         = yf.$(ClusterId).$(ProcId).out
output_destination = http://bsub.cern.ch:44444
transfer_input_files = root://eosuser.cern.ch//eos/user/b/bejones/condor/file.txt
requirements = OpSysAndVer =?= "CentOS8"
queue

exe:

[bejones@aiadm81 condor]$ cat yf.sh
#!/bin/bash

ClusterId=$1
ProcId=$2

echo "this is my first test" > first.${ClusterId}.${ProcId}.txt

echo "this is my second test" > second.${ClusterId}.${ProcId}.txt

cat file.txt > third.${ClusterId}.${ProcId}.txt

cat /etc/shadow

Web server log:

[root@bsub ~]# ./ws.py
serving at port 44444
128.142.130.177 - - [01/Feb/2022 10:53:07] "PUT /_condor_stderr HTTP/1.1" 201 -
128.142.130.177 - - [01/Feb/2022 10:53:07] "PUT /_condor_stdout HTTP/1.1" 201 -
128.142.130.177 - - [01/Feb/2022 10:53:08] "PUT /first.75.0.txt HTTP/1.1" 201 -
128.142.130.177 - - [01/Feb/2022 10:53:09] "PUT /second.75.0.txt HTTP/1.1" 201 -
128.142.130.177 - - [01/Feb/2022 10:53:10] "PUT /third.75.0.txt HTTP/1.1" 201 -

cheers,
Ben