Re: [HTCondor-users] Adding new ClassAd functions to HTC

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 11/11/2014 11:55 AM, Brian Bockelman wrote:
> > On Nov 10, 2014, at 3:56 AM, Fischer, Max (SCC) <max.fischer@xxxxxxx> wrote:
> >
> > Hi all,
> >
> > I’m looking for a way to query a REST service on our worker nodes for job matching. Dumping the information into the job/worker ClassAds as lists is not possible due to the data volume.
> > I’m constrained by having to execute the query on the worker node, but with information from the job, during START/RANK/… evaluation.
> >
> > It seems the only realistic solution is a custom ClassAd function, similar to the HDFS plugin [1]. I’m lost at how to include a new ClassAd function into HTC itself, though. Is there any tutorial, guide or other information on this?
>
> Hi Max,
>
> I think Dan Bradley wrote this up (very well) once - but I also think the tutorial disappeared when he moved on to a new job.  :(
>
> Here's a perhaps-simpler example custom function to start with:
>
> https://github.com/bbockelm/condor-ce/blob/master/src/classad_ce.cpp
>
> Basically, you compile your function into a shared library and add the shared library to CLASSAD_USER_LIB.
>
> One thing to consider is that the negotiator may evaluate the function many, many times (note - this means that having side-effects can be very bad!).  If you want an example of a custom function whose results are cached, look here:
>
> https://github.com/bbockelm/classad-xrootd-mapping/tree/master/src
>
> (Very similar idea to Stephen's HDFS function; I don't recall which code came first)
>
> > Alternatively, is there a simpler way to execute an arbitrary query from within a ClassAd?
>
> Nothing comes to mind.
>
> Brian
>
> PS - I'm working on a patch that allows you to write custom function in python, but it's not done / released yet.

FWIW, way back when we first added the ability to create custom classad 
functions in a shared library, we did so in order to address a 
job-steering-based-upon-data-placement scenario very similar to what is 
described above.  We wrote a paper on how it all worked, including a 
discussion about the custom classad function design; it may be 
interesting reading for you.

See:
Andrew Baranovski, Gabriele Garzoglio, Igor Terekhov, Alain Roy and Todd 
Tannenbaum, "Management of Grid Jobs and Data within SAMGrid", 
Proceedings of the 2004 IEEE International Conference on Cluster 
Computing, pages 353-360, San Diego, CA, September 2004.
  PDF: http://htcondor.org/doc/samgrid-cluster2004.pdf

BTW, you can find many more papers published by folks who worked on 
HTCondor over the years at:
  http://htcondor.org/publications.html

regards,
Todd