Re: [HTCondor-users] Using Multiple PRE_SKIP statements in .dag file

[Mark Coatsworth <coatsworth@xxxxxxxxxxx>]

HI Vinay, to answer your questions:

Multiple PRE_SKIP statements with different exit code: this is allowed, as long as you only use one PRE_SKIP statement per node in the dag. If you try adding more, the last one will override anything you've done previously. What you've done here looks fine.

As for ABORT-DAG-ON, the order of this declaration does not matter. If you add one of these rules for a node, and the node fails with the error code you specified, DAGMan will immediately stop its running jobs. So yes, it could be causing the premature exit of other nodes. But it would stop the dag and all its jobs entirely, not just one specific job.

What's the parent-child relationship between all the nodes you mentioned below?

Mark

On Tue, Apr 9, 2019 at 11:07 PM Vinay Kusuma <vinay@xxxxxxxxxxxxx> wrote:

Hello Everyone, 

Hope you all are doing well. I am Vinay, recently started working on HTC Condor as a part of my project. I want to skip few nodes depending on the input file provided, as of now i have written a pre script which checks my input and sends a exit status and this exit status is captured and used to skip or run a node in PRE_SKIP step.

The scenario is this, 

For a particular pipeline, I have three possible inputs. Either 'A' file, 'B' file or Both 'A' and 'B' file.

So, When 'B' file is the input, exit status given by pre script is 1

when 'A' files is the input, exit status given by pre script is 2

when both B' and 'A' files are used as input, exit status is 0.

The pre script name is check_input.py

My PRE SCRIPT and PRE_SKIP statements in .dag file are :-

SCRIPT  PRE     CREATE_B logs/check_input.py

SCRIPT  PRE     TARGET_B    logs/check_input.py

PRE_SKIP        CREATE_B 2

PRE_SKIP        TARGET_B    2

SCRIPT  PRE     PICK_A logs/check_input.py

SCRIPT  PRE     STAT_A logs/check_input.py

PRE_SKIP        PICK_A 1

PRE_SKIP        STAT_A 1

So, When i am running 'B' file as input, the exit status is 1. The nodes belonging to 'A' files in the above code as skipped as expected by exit status of 1 but when the execution reaches CREATE_B node, this node is exited by same exit status of 1.

So, After analysing the situation, I believe there can be two issues :-

1) Is it allowed to use multiple PRE_SKIP statements with different exit code as it did above?

2) Does the order of ABORT-DAG-ON of nodes have any impact on the premature exit of above node? 

Looking forward to your reply.

Thank you.

Regards,

Vinay Kusuma

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

--

Mark Coatsworth

Systems Programmer

Center for High Throughput Computing

Department of Computer Sciences

University of Wisconsin-Madison