Re: [Condor-users] about different dagman jobs problem

["R. Kent Wenger" <wenger@xxxxxxxxxxx>]

On Thu, 1 Oct 2009, Tao.3.Chen@xxxxxxxxxxxxxxxxxxxxxxxxxxx wrote:

>     I am using DAGMan to submit specific order jobs, I submit 2 different
> dagman jobs to same executor PC(I have only one executor) :
>
> dag1:
> JOB testA testA.sub
> JOB testB testB.sub
> JOB testC testC.sub
> PARENT testA testB CHILD testC
>
> dag2:
> JOB testD testD.sub
> JOB testE testE.sub
> JOB testF testF.sub
> PARENT testD testE CHILD testF
>
> But the result is: the executor will always execute
> testA,testB,testD,testE,and then testC,testF, it is not what I want. I
> need condor first finish dag1,then dag2,not like first all parent jobs
> testA,testB,testD,testE, then child jobs testC,testF, does anyone have
> this kind of problem? How can I solve this problem?
> Thank you in advance!

Remember that there is no order of jobs guaranteed between separate DAGs.

Basically, you have two choices: make everything into one big DAG, or 
create a "parent" DAG that runs the two sub-DAGs.

If you make everything into once big DAG, you would do the following in
a single DAG file:

JOB testA testA.sub
JOB testB testB.sub
JOB testC testC.sub

JOB testD testD.sub
JOB testE testE.sub
JOB testF testF.sub

PARENT testA testB CHILD testC
PARENT testC CHILD testD testE
PARENT testD testE CHILD testF


If you want a parent DAG, you'd do the following:

dag_parent:

SUBDAG EXTERNAL first dag1
SUBDAG EXTERNAL second dag2
PARENT first CHILD second

(You would need to keep your existing dag1 and dag2, and only submit 
dag_parent.)

You can also use the SPLICE keyword, which works similarly to SUBDAG 
EXTERNAL in terms of dependencies, but runs everything under a single
DAGMan instance:

dag_parent
SPLICE first dag1
SPLICE second dag2
PARENT first CHILD second

(Note that the above discussion assumes that you're running a pretty 
recent Condor -- 7.3.x or late 7.2.x.  If you have an older Condor 
version, you'll have to do things a bit differently.)

For more information, see sections 2.10.6.7 and 2.10.6.8 of the Condor 
manual:
http://www.cs.wisc.edu/condor/manual/v7.3/2_10DAGMan_Applications.html#SECTION003106700000000000000
http://www.cs.wisc.edu/condor/manual/v7.3/2_10DAGMan_Applications.html#SECTION003106800000000000000

Kent Wenger
Condor Team