Re: [HTCondor-users] DAGman VARS uses -append, bypassing submit conditionals

[Michael Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx>]

Consider an optional argument to the "VARS" command, in the same category as "ALL_NODES" or "RETURN" - a "PREPEND" argument at the end of the VARS statement, for example. Although there would have to be a consideration for backwards compatibility.. Does DAGman ignore statements it doesn't recognize?

Or maybe an "PREPEND_VARS" command with the same syntax as VARS?

But then, using a CONFIG would sidestep compat issues with the parser, and could be specified per-DAG. And also as a command-line option while we're at it. And by "we" I mean "you."

Michael V. Pelletier
Information Technology
Digital Transformation & Innovation
Integrated Defense Systems
Raytheon Company

-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of John M Knoeller
Sent: Tuesday, April 16, 2019 11:49 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: [External] Re: [HTCondor-users] DAGman VARS uses -append, bypassing submit conditionals

That's an interesting request.  What would you think about a dagman configuration option that says 

"pass VARS as a submit prefix statements  rather than as a submit append statements"

So you would be able to opt in to VARS as prefix, but not no a per-node basis.
would that work for you?

-tj

-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Michael Pelletier
Sent: Tuesday, April 16, 2019 9:55 AM
To: HTCondor-Users Mail List (htcondor-users@xxxxxxxxxxx) <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] DAGman VARS uses -append, bypassing submit conditionals

While working on a DAGman workflow to accelerate FPGA layout optimization, I came across an interesting issue.

Since there's only a handful of differences between different stages of the workflow, I set up a list of "if / endif" conditionals thinking that I could use a DAGman VARS statement to choose the right mode of operation of the shared submit description file.

However, DAGman appears to be set up to use the "-append" option for its "condor_submit" calls, which means that "MODE=1" is not set until after all of the if/endif statements have already been evaluated, and since the parser does only one pass on conditionals - as you would expect - the "-append" option means that DAGman VARS can't be used to control submit conditionals.

Is there a way to prepend VARS at submit time? If not, can we consider this a feature request?

I got around it by using a collection of stub submit files which set the MODE and then include the common submit, but it'd be nice to have one file instead of seven.

Michael V. Pelletier
Information Technology
Digital Transformation & Innovation
Integrated Defense Systems
Raytheon Company

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/