[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[HTCondor-users] Should we specify output/error files in transfer_output_files?



Dear HTCondor experts,

I hope you are doing well!
We currently have the following classad attributes in our submission script:

```
# Outputs: stdout, stderr, log
output = $(Cluster).$(Process).out
error = $(Cluster).$(Process).err
log = $(Cluster).$(Process).log

# No other files are to be transferred
transfer_output_files = ""

# Transfer outputs, even if the job is failed
should_transfer_files = YES
when_to_transfer_output = ON_EXIT_OR_EVICT
```

These options are used in 2 different contexts:

- using a remote schedd (grid sites): `condor_submit -terse -pool <remote_schedd>:9619 -remote <remote_schedd>` (expected to be equivalent to `-spool` for inputs/output files)

- using a local schedd (local cluster): `condor_submit -spool <job>`


To retrieve the output, we are just using `condor_transfer_data <cluster_id>.<process_id>`: - using a remote schedd, we get the `output`, `error` and `log` files in `initialdir`. - using a local schedd, we don't get anything and the command is not failing (`Fetching data files...`, `echo $?` gives me `0`). Is that expected?

Do you know why I can't get the output files using a local schedd? Is that because I have `transfer_output_files = ""`?

The documentation is not very clear about whether we should include the `output/error/log` values in `transfer_output_files`.
Do you have any recommendation?


Note: I am mostly using remote schedds, and I just interact with a single local schedd right now.

Thanks a lot,
Alexandre (for the LHCb experiment)