Re: [HTCondor-users] Determine when all jobs in a cluster have finished?

[Brian Candler <B.Candler@xxxxxxxxx>]

On Wed, Jan 30, 2013 at 09:21:46AM -0500, Brian Pipa wrote:
> I'd really like the whole thing to be self-contained in one DAG like:
> ###
> Job QueryDB querydb.job
> Job Workers workers.job
> Job PostProcess postprocess.job
> PARENT QueryDB CHILD Workers
> PARENT Workers CHILD PostProcess
> ###
> 
> since that seems much simpler and self-contained but I don't think
> that's doable since the results of the QueryDB job determines the data
> and number of worker jobs I'll need.

I thought I read something along those lines a while back. In principle I
think it is doable, because workers.job is not processed until the time that
job is actually submitted.

I don't know the cleanest way of making this dynamic, but you could try
simply overwriting the workers.job file from the querydb job.

Maybe what I am thinking of is this:
http://research.cs.wisc.edu/htcondor/manual/v7.8/2_10DAGMan_Applications.html#SECTION003107800000000000000
It says explicitly that a DAG may modify a sub-DAG before it is submitted.

Regards,

Brian.