Re: [HTCondor-users] Poll: Is embedding submit info into the start of a script of interest? (was Re: how does one get the filename automatically in condor?)

[michael.v.pelletier@xxxxxxxxxxxx]

It has always seemed to me that the script-comment directives were a result of a workaround for the design shortfall of their submission model, where thereâs nowhere except for the submit command line to specify a list of cryptic and inscrutable job parameters. The distinction between submission directives and the job itself is blurred as a result. Allowing the runtime script comments to specify job parameters enabled them to store the parameters in a single file, accomplishing what HTCondor had been able to do all along.

 

Comment-based also doesnât have the ability that you have in an âinclude commandâ directive to dynamically define job requirements.

 

 

Michael V Pelletier

Principal Engineer

Raytheon Technologies

Digital Technology

HPC Support Team

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Todd Tannenbaum
Sent: Tuesday, February 16, 2021 3:44 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Cc: Miranda, Brando <miranda9@xxxxxxxxxxxx>
Subject: [External] [HTCondor-users] Poll: Is embedding submit info into the start of a script of interest? (was Re: how does one get the filename automatically in condor?)

 

On 2/16/2021 1:11 PM, jfrey@xxxxxxxxxxx wrote:

You canât have a single file as both your submit description file and job executable, as you can with PBS or Slurm.

Hi folks,

I am wondering if our community is interested in seeing HTCondor have the ability for job submission information "embedded" at the start of a script PBS-style, like Brando shows below?   If yes, please help me to understand how/why you would find this desirable.  I do not want to miss out on an opportunity to make HTCondor easier to use.

I personally find having my  job submission directives (submit file) and my job itself (executable) kept separated into two different files to be a more clean solution.  But as Brando is not the first person to ask about the below, I cannot help but wonder if I am missing something here. 

Thanks all,
Todd

On Feb 16, 2021, at 11:06 AM, brando.science@xxxxxxxxx wrote:

 

Apologies I don't think I understand your response. I usually have a submission script as job.sub that specifies the executable path. If I could instead use the actually executable script as the submission script that would be great too e.g. I usually do that with qsub as follows:

```

#!/homes/miranda9/.conda/envs/automl-meta-learning/bin/python

#PBS -V

#PBS -M me@xxxxxxxxx

#PBS -m abe

#PBS -lselect=1:ncpus=112

import sys

import os

for p in sys.path:

    print(p)

print(os.environ)

 

-- 

Todd Tannenbaum <tannenba@xxxxxxxxxxx>  University of Wisconsin-Madison

Center for High Throughput Computing    Department of Computer Sciences