Re: [HTCondor-users] Determine when all jobs in a cluster have finished?

[Michael Aschenbeck <m.g.aschenbeck@xxxxxxxxx>]

If you know the cluster ID, you do something like this (taken directly from the manual: http://research.cs.wisc.edu/htcondor/manual/current/condor_wait.html)

"
condor_wait logfile 40

This command waits for all jobs that exist in logfile with a job ClassAd attribute ClusterId of 40 to complete.
"

I have code that does something similar but i just wait for any job to get back to post process it.  I utilize "condor_wait -num" (also described int he manual).  In that case, i have a loop that is something like this (in psuedo-ish c++)

submitAllJobs(); // use MyCondorLog.txt as the log
for(int i=0;i<numJobs;i++)
{
    string command = "condor_wait " + intToString(i) + " MyCondorLog.txt";

    system(command);

    postProcess(getCluster(i));
}

All jobs will be running, when the first one comes back, it post processes it and continues in that fashion.  If 5 jobs have finished and your code calls condor_wait -num 2, it does not "wait" at all since 5 jobs have already come back and you are telling it to wait until 2 jobs are done.

On Thu, Jan 31, 2013 at 11:09 AM, Brian Pipa <brianpipa@xxxxxxxxx> wrote:

On Thu, Jan 31, 2013 at 1:01 PM, Brian Pipa <brianpipa@xxxxxxxxx> wrote:
> The only tricky part I can think of is QueryDB knowing when each job
> is finished and to know when to stop looking for results (so it
> doesn't wait forever and knows when it's done). I guess I'll just have
> to periodically run condor_q and condor_query to figure out which
> worker jobs are done and once I know they are done, postprocess them.
>

Edit: that should have said condor_history not condor_query. But...I
just saw the condor_wait part of the reply - that should do nicely to
tell which workers are done... is there a way to know when all of the
workers in a cluster are done? I guess that's where I would use
condor_q <clusterid> instead.

Brian
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/