Re: [Condor-users] Need some guidelines for a chunking upvanilla Win appplication across nodes

["Ian Chesal" <ICHESAL@xxxxxxxxxx>]

>	I have this all working great. It seems an obvious question, but
the  
> last step in my process is to run a program that collates the  
> results.This cannot run until all jobs are complete and the output  
> files are back at the submitting machine. How does one know this  
> apart from watching condor_q

If you're scripting the submission and post-processing portion and you
don't need a lot of fancy behaviour you can call the condor_wait command
and it will watch the local log file for the cluster and not return
until all the jobs in the cluster have completed. It has a couple of
options for controlling verbosity and a wait-no-longer-than time.

If you want to do anything more complicated, like perhaps performing an
action when every job completes, you can look into the Condor.pm module
found in the condor/lib directory. It lets you read a log file and
register call backs when certain actions are spotted in the log file.
That's a good place to start learning about how to parse the log file.

And finally you can look at wrapping the output from condor_q. Beware
that calling condor_q frequently and from many machines in parallel can
degrade your scheduling performance as it interrupts the scheduling
thread. Not a big deal if you have a distributed scheduler setup, but it
will have a noticeable impact if you have a system where there's only a
handful of central schedulers.

- Ian