Re: [HTCondor-users] Help with software RASPA on HTCondor

[Bruno Luciano Amadio Caires <brunolac@xxxxxxxxxx>]

Hi Beyer,

Thank you for your help :)

Em sex., 30 de ago. de 2024, 03:13, Beyer, Christoph <christoph.beyer@xxxxxxx> escreveu:

Hi,

let me look that up in the manual for you ;)

If the file transfer mechanism is enabled, HTCondor will transfer the following files from the execute machine back to
the submit machine after the job exits, as the output sandbox.
1. the output file, as defined with the output command
2. the error file, as defined with the error command
3. any files created by the job in the remote scratch directory.
A path given for output and error commands represents a path on the submit machine. If no path is specified, the
directory specified with initialdir is used, and if that is not specified, the directory from which the job was submitted
is used. At the time the job is submitted, zero-length files are created on the submit machine, at the given path for the
files defined by the output and error commands. This permits job submission failure, if these files cannot be written
by HTCondor.
To restrict the output files or permit entire directory contents to be transferred, specify the exact list with trans-
fer_output_files . When this comma separated list is defined, and any of the files or directories do not exist as the
job exits, HTCondor considers this an error, and places the job on hold. Setting transfer_output_files to the empty
string (ââ) means no files are to be transferred. When this list is defined, automatic detection of output files created by
the job is disabled. Paths specified in this list refer to locations on the execute machine. The naming and placement of
files and directories relies on the term base name. By example, the path a/b/c has the base name c. It is the file name
or directory name with all directories leading up to that name stripped off. On the submit machine, the transferred files
or directories are named using only the base name. Therefore, each output file or directory must have a different name,
even if they originate from different paths.

Best

christoph

--
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx

---

Von: "Bruno L Amadio Caires" <bruno@xxxxxxxxxxxxx>
An: "Todd Tannenbaum" <tannenba@xxxxxxxxxxx>, "HTCondor-Users Mail List" <htcondor-users@xxxxxxxxxxx>
Gesendet: Freitag, 30. August 2024 03:38:40
Betreff: Re: [HTCondor-users] Help with software RASPA on HTCondor

Hello Todd,

It worked when I added the "getenv = True" option in the submission file. I was trying "environment" option to indicate the directory where the software is installed.

Another question: I had to explicitly specify the directories that should be transferred back after execution with the line:

transfer_output_files = Output, Movies, Restart, VTK

Shouldn't the following options be sufficient to ensure that all output files are transferred back?

should_transfer_files = YES
when_to_transfer_output = ON_EXIT

Thank you for your help.

Em 29/08/2024 18:59, Todd Tannenbaum escreveu:

On 8/28/2024 6:56 PM, Bruno L Amadio Caires wrote:

Hello,

I have HTCondor running in my environment, and I've successfully tested it with several software applications so far.

However, I'm currently encountering an issue with running the RASPA software. While RASPA is installed on the execution machine and runs perfectly when executed locally, it fails to produce any results when submitted through HTCondor. Moreover, the HTCondor log file "RASPA_error.err" does not indicate any errors, yet the expected output files from the RASPA execution do not appear on the HTCondor server.

Could you provide any insights or suggestions on how to resolve this issue?

Hi Bruno,

If I had to guess, there is some missing environment variable. You could try adding "getenv=true" to your submit file below and try again.

Another idea is doing "condor_submit -i <name of submit file>", which should drop you at an interactive ssh prompt in a slot that matches your requirements (HAS_RASPA=True) after your input files are transferred, with the same environment values. Then you could issue the command "/usr/local/bin/simulate" and debug the problem interactively which may make things much easier.

Let us know how it goes!

regards,
Todd

https://iraspa.org/download/raspa-manual-23-may-2021/

My submission file

-------------------

universe = vanilla
executable = /usr/local/bin/simulate
arguments = -i simulation.input -d /usr/local/
output = RASPA_output.out
error = RASPA_error.err
log = RASPA_job.log
environment = "RASPA_DIR=/usr/local"
transfer_input_files = run, simulation.input
should_transfer_filesÂÂ = YES
when_to_transfer_output = ON_EXIT
transfer_executable = False
requirements = HAS_RASPA

queue

--------------------

Thank you.

-- 
Bruno L. Amadio Caires

InformÃtica - FT

Universidade Estadual de Campinas

-- 
Bruno L. Amadio Caires

InformÃtica - FT

Universidade Estadual de Campinas

Fone / WhattsApp:(19)2113-3537 

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/