Mailing List Archives

Authenticated access

 		[Computer Systems Lab]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] MATLAB licenses

We've had good luck with both Octave and Matlab here in Madison.

The Matlab compiler only uses a license for creating the job and after
that it leave the license pool alone. So as many jobs as are needed and as one 
has priority to run can be handled this way. In Madison we keep the runtime
readable by all in AFS and setup the path and environment as needed
in the job and only have to transport the compiled job and the data.

We've had some luck using Octave. One advantage other then license
issues with Octave is that with care it can be condor_compiled allowing
us to run the jobs in "Standard" universe. We find that "Vanilla" jobs which can't be checkpointed 
have poor throughput on non-dedicated nodes and pools when they get up
around an hour in length.

I'd be happy to send some sample scripts for those interested.

Bill

Bruno Goncalves wrote:
One way would be to allow just N jobs to run at the same time (N being the number of licenses you have available). You could do this by specifying on the submit file that you only want specific machines to run those jobs. For instance, if you have only two licenses, you could do:
Requirements = (Machine=="machine1.mydomain.com <http://machine1.mydomain.com>" || Machine=="machine2.mydomain.com <http://machine2.mydomain.com>")
So you would only run two jobs at once, but wouldn't have to worry about the licensing issue.
On 2/9/06, *f.pappenberger@xxxxxxxxxxxxxxx <mailto:f.pappenberger@xxxxxxxxxxxxxxx>* <f.pappenberger@xxxxxxxxxxxxxxx <mailto:f.pappenberger@xxxxxxxxxxxxxxx>> wrote: 

    Our solution is  a license for the Matlab compiler toolbox. We
    submit the compiled  programs to condor. Is still a sligth pain as
    you have to make shure that you take all the libraries with you
    (but perhaps someone has a better solution for that)

    flo

    In message <20060209012911.16329.qmail@xxxxxxxxxxxxxxxxxxxxxxx
    <mailto:20060209012911.16329.qmail@xxxxxxxxxxxxxxxxxxxxxxx>>
    Condor-Users Mail List < condor-users@xxxxxxxxxxx
    <mailto:condor-users@xxxxxxxxxxx>> writes:
    > --0-307217929-1139448550=:14483
    > Content-Type: text/plain; charset=iso-8859-1
    > Content-Transfer-Encoding: 8bit
    >
    > well most people in my lab use matlab.
    >
    >   so i have to use it in any case.
    >
    >   plus matlab is not installed on every system. but only one
    system from where users access it and use it over the network.
    >
    >   all the processing is done on the system where it is installed.
    >
    >   so how will condor schedule a matlab script in this case?
    >
    >
    > Maxim Kovgan <maxim.kvg@xxxxxxxxx <mailto:maxim.kvg@xxxxxxxxx>>
    wrote:  Another thing... are you sure matlab is the only tool that
    can run
    > your scripts ?
    > I mean if you use some basic features of matlab, maybe octave will
    > suffice, and it has no licensing issues.
    >
    > Max.
    >
    >
    > On 2/9/06, Maxim Kovgan  wrote:
    > > Hi,
    > > I can think of 2 directions:
    > > 1. you can wrap your matlab script into another script, which will
    > > e.g. check how many licenses are left and poll the license server
    > > until some threshold is reached, prior to launching matlab
    executable.
    > >
    > > 2. you can try using limits of maximum running jobs via the
    pool global macro:
    > > MAX_JOBS_RUNNING, but then you need to make sure ... the only
    matlab
    > > user runs this job.
    > >
    > > 3. If it is possible,  you should obtain enough licenses to
    have the
    > > whole pool busy.
    > >
    > >
    > > --Max.
    > >
    > >
    > > On 2/9/06, Ben Liblit
    >  wrote:
    > > > I recently submitted a batch of 260 jobs, each of which runs
    two MATLAB
    > > > batch scripts ("matlab -nojvm -nodisplay -r ...") toward the
    end of its
    > > > work.  53 of these jobs failed because they were unable to
    obtain a
    > > > MATLAB license:
    > > >
    > > >      License Manager Error -4.
    > > >      Maximum number of users for MATLAB reached.
    > > >      Try again later.
    > > >      To see a list of current users use the lmstat utility.
    > > >
    > > > The MATLAB scripts are short, each lasting no more than a
    few seconds.
    > > > But my 260 jobs include many subgroups of about equal
    duration, which
    > > > means I'll be creating large spikes in MATLAB license demand.
    > > >
    > > > What's the right approach here?  Is there a different way
    one is
    > > > supposed to run MATLAB under Condor?  Is there a way to
    declare the need
    > > > for a MATLAB license as part of a job's classad
    restrictions?  Should
    > > > every Condor node have a specially reserved MATLAB license
    for the
    > > > exclusive use of the Condor job it is currently running, or
    multiple
    > > > such licenses for nodes that run multiple concurrent jobs?
    > > >
    > > >
    > > > _______________________________________________
    > > > Condor-users mailing list
    > > > Condor-users@xxxxxxxxxxx <mailto:Condor-users@xxxxxxxxxxx>
    > > > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
    <https://lists.cs.wisc.edu/mailman/listinfo/condor-users>
    > > >
    > >
    >
    > _______________________________________________
    > Condor-users mailing list
    > Condor-users@xxxxxxxxxxx <mailto:Condor-users@xxxxxxxxxxx>
    > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
    >
    >
    >
    > Junaid N. Sahibzada
    > Cell # (+61) 404 998 494
    > 284/9 Crystal St. Waterloo, 2017, NSW, Australia
    > International Student MSc Internetworking, UTS, Australia
    > Bachelor of Information Technology, NUST, Pakistan
    >
    >
    >
    >
    >
    >
    >
    >
    >
    >
    >
    >
    >
    >
    >
    > ---------------------------------
    > Brings words and photos together (easily) with
    >  PhotoMail  - it's free and works with Yahoo! Mail.
    > --0-307217929-1139448550=:14483
    > Content-Type: text/html; charset=iso-8859-1
    > Content-Transfer-Encoding: 8bit
    >
    > well most people in my lab use matlab.<br>  <br>  so i have to
    use it in any case.<br>  <br>  plus matlab is not installed on
    every system. but only one system from where users access it and
    use it over the network.<br>  <br>  all the processing is done on
    the system where it is installed.<br>  <br>  so how will condor
    schedule a matlab script in this case?<br>  <br><br><b><i>Maxim
    Kovgan < maxim.kvg@xxxxxxxxx <mailto:maxim.kvg@xxxxxxxxx>></i></b>
    wrote:<blockquote class="replbq" style="border-left: 2px solid
    rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;">  Another
    thing... are you sure matlab is the only tool that can run<br>your
    scripts ?<br>I mean if you use some basic features of matlab,
    maybe octave will<br>suffice, and it has no licensing
    issues.<br><br>Max.<br><br><br>On 2/9/06, Maxim Kovgan < maxim.kvg
    @gmail.com=""> wrote:<br>> Hi,<br>> I can think of 2
    directions:<br>> 1. you can wrap your matlab script into another
    script, which will<br>> e.g. check how many licenses !
    >  are left
    >  and poll the license server<br>> until some threshold is
    reached, prior to launching matlab executable.<br>><br>> 2. you
    can try using limits of maximum running jobs via the pool global
    macro:<br>> MAX_JOBS_RUNNING, but then you need to make sure ...
    the only matlab<br>> user runs this job.<br>><br>> 3. If it is
    possible,  you should obtain enough licenses to have the<br>>
    whole pool busy.<br>><br>><br>> --Max.<br>><br>><br>> On 2/9/06,
    Ben Liblit <liblit @ cs.wisc.edu=""> wrote:<br>> > I recently
    submitted a batch of 260 jobs, each of which runs two MATLAB<br>>
    > batch scripts ("matlab -nojvm -nodisplay -r ...") toward the end
    of its<br>> > work.  53 of these jobs failed because they were
    unable to obtain a<br>> > MATLAB license:<br>> ><br>>
    >      License Manager Error -4.<br>> >      Maximum number of
    users for MATLAB reached.<br>> >      Try again later.<br>>
    >      To see a l!
    >  ist of
    >  current users use the lmstat utility.<br>> ><br>> > The MATLAB
    scripts are short, each lasting no more than a few seconds.<br>> >
    But my 260 jobs include many subgroups of about equal duration,
    which<br>> > means I'll be creating large spikes in MATLAB license
    demand.<br>> ><br>> > What's the right approach here?  Is there a
    different way one is<br>> > supposed to run MATLAB under
    Condor?  Is there a way to declare the need<br>> > for a MATLAB
    license as part of a job's classad restrictions?  Should<br>> >
    every Condor node have a specially reserved MATLAB license for
    the<br>> > exclusive use of the Condor job it is currently
    running, or multiple<br>> > such licenses for nodes that run
    multiple concurrent jobs?<br>> ><br>> ><br>> >
    _______________________________________________<br>> >
    Condor-users mailing list<br>> > Condor-users@xxxxxxxxxxx
    <mailto:Condor-users@xxxxxxxxxxx><br>> >
    >  https://lists.cs.wisc.edu/mailman/listinfo/condor-users <br>>
    ><br>><br><br>_______________________________________________<br>Condor-users
    mailing list<br>Condor-users@xxxxxxxxxxx
    <mailto:Condor-users@xxxxxxxxxxx><br>
    https://lists.cs.wisc.edu/mailman/listinfo/condor-users<br></liblit></maxim.kvg></blockquote><br><BR><BR><DIV>

    > <DIV>
    > <DIV>
    > <DIV>
    > <DIV>
    > <DIV>
    > <DIV>
    > <DIV>
    > <DIV>
    > <DIV>
    > <DIV>
    > <DIV>
    > <DIV>
    > <DIV><STRONG>Junaid N. Sahibzada</STRONG></DIV>
    > <DIV><STRONG>Cell # (+61) 404 998 494 </STRONG></DIV>
    > <DIV><STRONG>284/9 Crystal St. Waterloo, 2017, NSW,
    Australia</STRONG></DIV>
    > <DIV><STRONG>International Student MSc Internetworking, UTS,
    Australia</STRONG></DIV>
    > <DIV><STRONG>Bachelor of Information Technology, NUST,
    Pakistan</STRONG></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV><p>

    >               <hr size=1>Brings words and photos together
    (easily) with<br>
    > <a
    href="http://us.rd.yahoo.com/mail_us/taglines/PMDEF3/*http://photomail.mail.yahoo.com
    <http://us.rd.yahoo.com/mail_us/taglines/PMDEF3/*http://photomail.mail.yahoo.com>">PhotoMail
    </a> - it's free and works with Yahoo! Mail.
    > --0-307217929-1139448550=:14483--
    _______________________________________________
    Condor-users mailing list
    Condor-users@xxxxxxxxxxx <mailto:Condor-users@xxxxxxxxxxx>
    https://lists.cs.wisc.edu/mailman/listinfo/condor-users




--
*******************************************
Bruno Miguel Tavares Goncalves, MS
PhD Candidate
Emory University
Department of Physics
Office No. N117-C
400 Dowman Drive
Atlanta, Georgia 30322
Homepage: www.bgoncalves.com <http://www.bgoncalves.com>
Email: bgoncalves@xxxxxxxxx <mailto:bgoncalves@xxxxxxxxx>
Phone: (404) 712-2441
Fax:   (404) 727-0873
*******************************************
------------------------------------------------------------------------

_______________________________________________
Condor-users mailing list
Condor-users@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/condor-users