Re: [HTCondor-users] Avoiding combinatorial explosion in dependencies between spliced DAGS

["R. Kent Wenger" <wenger@xxxxxxxxxxx>]

On Thu, 30 Jul 2015, John N Calley wrote:

> What I'd really like to do is to reach 'into' each sub-dag and insert 
> dependencies between specific final nodes and specific initial nodes. 
> I've considered hacking this solution together, but the ways of doing it 
> that I can think of seem inelegant. I wonder if anyone has thoughts on 
> how to do this kind of thing cleanly? To expand a bit, this comes up 
> when I want to do Analysis A on samples 1-2000 and then I want to do 
> Analysis B on the same samples. Analysis B for sample 1 depends on 
> Analysis A for the same sample, but not on Analysis A for any other 
> samples. It's a shame to require that Analysis A finish for all samples 
> before I start Analysis B for any samples, but that is what I feel stuck 
> with at the moment.

I thought that the following would work:

  * top.dag
  splice A A.dag
  splice B B.dag
  parent A+A1 child B+B1

  * A.dag
  job A1 ...

  * B.dag
  job B1...

Unfortunately, it doesn't.  I think that the nodes from the splices are 
not fully incorporated into the top-level DAG at the time that the 
parent...child line is parsed.

At any rate, I've created a ticket for this request:

  https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=5213

and I'll be coming up with a design document soon.

Kent