Re: [condor-users] condor in the CG industry - maya, emntal ray & PRMan

["Michael S. Root" <mike@xxxxxxxxxxxxxx>]

Hi Marc.  We just started using Condor here at Tweak Films a couple of 
months ago to replace a really lame script-based system I'd thrown 
together while doing a hundred other things.  It's taken a bit of fussing 
around to get Condor to work in our pipeline, but now that we understand 
it better, it works pretty well.  

We've just (almost) completed our first show using it, and we've run well 
over 200,000 jobs, using over 2 1/2 years worth of CPU time since Feb. 11, 
and it handled the load with only a couple of minor problems.

I ended up writing a Python module that generates all the submit files, log 
file locations, output/error file locations, etc., thus isolating our 
artists from all that stuff.  We then have a relatively small set of 
command line tools that all use that same Python module for submitting 
Entropy (essentially the same as PRMan), Shake, Maya, and other jobs.  I 
also made an internal web page that shows the status, output, and error 
logs of all running jobs, and the status of all the machines.  We still 
rely on out-of-the-box Condor tools for removing jobs, changing 
priorities, etc.

You should check the condor-users mailing list archive for a question I 
asked about pre-emption and job scheduling.  Specifically, changing the 
following:

START = $(START) && ( State != "Claimed" || $(StateTimer) < 20)
PREEMPT = FALSE
PRIORITY_HALFLIFE = 5
NEGOTIATOR_INTERVAL = 5

made all the difference in making Condor schedule jobs in a manner than fit 
our needs.  This way, Condor tries to always give users the same number of 
resources (assuming user priorities are the same), regardless of previous 
usage, but will never kill jobs that are already running.  We do pay a 
per-job overhead penalty for doing it this way, but by setting the 
NEGOTIATOR_INTERVAL to 5 seconds, it's fairly minimal.  Most of our tools 
also have the ability to run multiple frames per Condor job, which makes 
the overhead even less of an issue.

Anyway, to make a long story short, it works great for us, and you can't 
beat the price!

-Mike
mike@xxxxxxxxxxxxxx


On Monday 12 April 2004 01:36, Marc Anthony Pierre Barrette wrote:
> hello,
>
>    is condor beeing used in the Computer Graphics industry for large
> scale render farms? specifically for various render engines ie. PRMan,
> Mental Ray, Maya. If anyone could point me in some direction it would be
> greatly appreciated, and examples are always nifty.
>
>    thanks in advance

Condor Support Information:
http://www.cs.wisc.edu/condor/condor-support/
To Unsubscribe, send mail to majordomo@xxxxxxxxxxx with
unsubscribe condor-users <your_email_address>