Re: [Condor-users] Need some guidelines for a chunking up vanilla Win appplication across nodes

["Jason Stowe" <jstowe@xxxxxxxxxxxxxxxxxx>]

Andrew,
There are a couple of ways to handle this issue, depending upon the
nature of your pool of resources, and the solver itself. If you have a
heterogeneous pool of workstations, with variability in their speed
and availability, and your solver computation is about the same for
each frequency, I would recommend a different tack.

Instead of having each job run some number of atomic units of work, a
good practice would be to have a job for each atomic unit of work.
Thus if you have 1000 frequencies, create a job file that does:
solve -start #n -end #n file.in file.out

Your submit file would look something like:
Executable = solve
Args = -start $(Process) -end $(Process) file.in file.out
Universe = vanilla
#other job attributes
out = frequency.$(Process).out
error = frequency.$(Process).error
Log = run.log

Initialdir = frequency_$(Process)
queue <#frequencies>

You can auto-generate this file and insert the <#frequencies> then
submit to Condor. The $(Process) will put the number of the process,
which ranges from 0 to n-1. You could alternately create a
ProcessPlusOne attribute if necessary. In this way, if any of your
processes get interupted you lose the minimal amount of work, and if
some machines process faster they'll get more jobs. This should lead
to efficient execution for as many nodes you have.

Hope this helps,
Jason


--

===================================
Jason A. Stowe

Phone: 607.227.9686
jstowe@xxxxxxxxxxxxxxxxxx

Cycle Computing, LLC
http://www.cyclecomputing.com


On 2/23/07, Andrew Cunningham <andrewc@xxxxxxx> wrote:
> I am a very new user for Condor ( Windows)
> So this is pretty basic question...
>
> I have a vanilla "solver" type application that operates on a set of
> n frequencies 1...n
> e.g.
> % solve -start 1 -end 10 file.in file.out
> solves from frequency 1 to frequency 10
>
> So if I was to run on 2 nodes, the first node would be run
> % solve -start 1 -end 5 file.in file1.out
> and the second
> % solve -start 6 -end 10 file.in file2.out
> etc...
>
> Up to a maximum of n nodes
>
> When done, I would transfer the files back , then run a utility to
> combine the results.
>
> 1. Is there some way to use Condor to generate the -start and -end
> parameters based on the number of available nodes?
> 2. I also need to run a utility to get 'n' ( number of frequencies)
> as that is encoded in the input file. I  would then need pass that
> somehow to Condor (assuming it can do 1. )
>
>
> Andrew
>
>
>
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at either
> https://lists.cs.wisc.edu/archive/condor-users/
> http://www.opencondor.org/spaces/viewmailarchive.action?key=CONDOR