Hi Ian, that was me. We can take this offline and I'll email you my
example implementation directly.
Cheers,
Mark
Dr Ian C. Smith wrote:
Dear All,
I was at the Condor Week in Edinburgh a while back and I
remember that a talk on the eMinerals project mentioned
something about using recursive DAGs to resubmit long
running jobs automatically. I think this would be really
useful here but I can't seem to find any info on how to
do it.
To give you some background. Our Condor pool is only available
for around 16 hours each day so jobs needing to run for
several days need to have some way of getting the intermediate
results back and then using them as input on the next run.
At present we can look at the system clock to see if time is
about to run out then exit gracefully so that Condor stages the
results back. The submit host needs a cron style script to
re-submit the job which is messy to say the least.
One idea is to have the job run only for a few hours only at a time.
If it's a time stepping code we can easy specify the number of
steps to be taken and the solution can be picked again later were we
left off. If it gets killed then a couple of hours of wallclock
have been wasted - no big deal. I imagine that this could easily
be achieved with a DAG but of course we don't know how many time steps
are needed in total a priori. This is were a recursion would be useful if
only I could work out how to do it.
Anyone have any ideas ??
cheers,
-ian.
PS pedantic point: doesn't a recursion introduce a cyclic dependency ?
Why not just allow cycles in the DAG dependency graph ?
_______________________________________________
Condor-users mailing list
Condor-users@xxxxxxxxxxx
http://lists.cs.wisc.edu/mailman/listinfo/condor-users
--
Dr Mark Calleja
Cambridge eScience Centre, University of Cambridge
Centre for Mathematical Sciences, Wilberforce Road, Cambridge CB3 0WA
Tel. (+44/0) 1223 765317, Fax (+44/0) 1223 765900
http://www.escience.cam.ac.uk/~mcal00
---------------------------------------------
Department of Earth Sciences
University of Cambridge
Downing Street
Cambridge CB2 3EQ
Phone: ( +44 ) 1223 333400
Fax: ( +44 ) 1223 333450
_______________________________________________
Condor-users mailing list
Condor-users@xxxxxxxxxxx
http://lists.cs.wisc.edu/mailman/listinfo/condor-users
![[Computer Systems Lab]](http://www.cs.wisc.edu/pics/csl_logo.gif){kind=logo}