[Condor-users] DAGman: array dependencies, dynamic DAGs

[Mark Visser <markv@xxxxxxxxxxxxxx>]

Hi folks, I just joined the list, and want to introduce myself. I was (I 
think) the first Condor user in the visual effects world back in 1998. I 
haven't been involved in Condor since 2004, but I'm back... and wow, you 
guys have been busy! I'm now at a small VFX studio in Montreal... we've 
jumped ship from a proprietary grid solution and I'm again looking at 
integrating Condor into a VFX workflow. I see consumable resources are 
now supported natively... cool. I have a couple questions about DAGman, 
though:

First, does DAGman have a way to declare dependencies between clusters 
of equal numbers of jobs, but have the dependencies actually be between 
jobs? Sun Grid Engine calls this an "array dependency". For example, 
given cluster A of 100 jobs, and cluster B of 100 jobs, I want to define 
a relationship between A and B such that:
 B.0 depends on A.0
 B.1 depends on A.1
 B.2 depends on A.2
 ...
 B.99 depends on A.99

This obviously requires that either A and B have the same number of 
jobs, or that the dependencies between A's jobs and B's jobs are 
explicitly described.

Second question: Is there any way to add or modify dependencies after a 
DAG is submitted? For example, I submit long-running cluster A, then 
several hours later I want to submit cluster B, such that B depends on 
the successful completion of A. Does DAGman re-read its dag file, or is 
it read only once?

cheers,
-Mark

P.S. Hey Todd!

--
Mark Visser, Software Director
Lumière VFX
Email: markv@xxxxxxxxxxxxxx
Phone: +1-514-316-1080 x3030