Re: [HTCondor-users] Determine when all jobs in a cluster have finished?

[Michael Aschenbeck <m.g.aschenbeck@xxxxxxxxx>]

I believe you are looking for "condor_wait".  The following page has all the info you need.

http://research.cs.wisc.edu/htcondor/manual/current/condor_wait.html

On Tue, Jan 29, 2013 at 3:18 PM, Brian Pipa <brianpipa@xxxxxxxxx> wrote:

Short: I'm trying to figure out when all jobs from a job cluster have
finished so that I can do some post-processing. I can think of lots of
ways for me to code this up, but it seems like there would be some
easy way in Condor to do this - does anyone know how?

Long: I have a single Java master task (that is also a Condor job,
though that's not relevant) that does a large DB query then splits the
results into chunks and submits each chunk to Condor as a job via one
ClassAd so they all have the same Cluster id. These jobs are all Java
worker jobs that call various tools to process the data. I have all of
the output for each worker cluster going to a single directory so it's
easy to keep them together and know what output is from which run. As
I said above, I can think of a bunch of ways I could code up a
solution but it seems like Condor might have a way to tell if a
Cluster of jobs has finished or not.  Does anyone know if Condor does
have a way to do this?

UPDATE: while typing this email up I found:
condor_q <cluster>
which might work. When I submit the one big worker job, I capture the
output from condor_submit and I can parse out the id from that "X
job(s) submitted to cluster Y".  Then, after I submit the job, I can
call
condor_q Y
periodically until it tells me no more jobs are in the q.
or I could call
condor_q Y |grep Y
until I get nothing back.

Does this sounds right/make sense? is there an easier way to do this?
My way seems kind of hacky though I think it should work.

Thanks!
Brian
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/