Re: [Condor-users] Visually design Condor DAGs

[Jan Ploski <Jan.Ploski@xxxxxxxx>]

condor-users-bounces@xxxxxxxxxxx schrieb am 02/26/2008 05:01:53 PM:

> Hi,
> 
> Thanks for your reply. Actually, I was trying to identify if there
> exists a solution to do it or if it would be otherwise interesting to
> have one. I am working on a thesis in the area of grid computing and I
> am trying to identify candidate developments that I could undertake
> while adding value to the grid community.
> 
> I have seen some workflow solutions - the latest one I've been playing
> with is Taverna - that allow people with not so much computing
> background to design experiments, and at the same time I found that
> Condor's handling of DAGs seems to be quite mature, so I figured it
> might make sense to make some kind of mix between those approaches.
> 
> I understand that you can have lots of DAGs, though if they all
> corresponded to a same 'template' like in a parameter sweep
> application, a way to design generic template workflows could perhaps
> still add to the subject. On the other hand, if there are lots of DAGs
> because they correspond to completely different schemes of execution,
> they would have to be individually designed, but that I think would
> still make sense because we need to think about those schemes
> individually anyway.

Hi,

In my particular case the DAGs are simple sequential executions of 4 
Condor-G jobs with POST-scripts to pin down the jobs to a particular site, 
to check the results, and to retry on failure, so the visual design would 
likely add little. However, I understand that you might theoretically run 
into a more complicated structure where the visual approach might be 
worthwhile - this would especially be the case if you had reusable jobs, 
but there could be different possible dependencies between them (or nested 
DAGs, in that matter). Even in such case I believe that easy 
parametrization (when creating instances of the workflows) would be 
necessary and might be challenging for GUI design.

My suggestion would be to find the application scenarios first and move on 
from there. Maybe you could "port" scenarios of Taverna users? So far, I 
have read about quite a few "Grid workflow" projects, but have not seen so 
many actual Grid workflow examples. These projects are founded on the 
premise that "users need shiny, service-oriented, generic workflow tools", 
which is rather fuzzy - seldom do they demonstrate how concrete, 
particular scenarios will benefit from the tools or contrast them with 
alternatives like application programmers doing the necessary custom 
programming on the users' behalf (which I suspect is how things get done 
in reality).

> This is to say that I don't have an actual application scenario,
> rather I am trying to see if there exists one that would justify a
> development like the one I mentioned, or a related one (and whether
> there are mature solutions already).

I hope you will find a good one! Certainly, Condor provides a very 
flexible job execution environment. In fact, so dynamic and flexible that 
it may turn out to be another challenge if one's thoughts are confined to 
diagram painting (POST-scripts updating job command files, periodically 
evaluated expressions that may influence execution, ...)

Regards,
Jan Ploski