Re: [Condor-devel] Fwd: RFC: Parallel classad evaluations

[Jon Thomas <jthomas@xxxxxxxxxx>]

On Sat, 2012-04-07 at 20:15 -0500, Ziliang Guo wrote:
> Forwarded from the FW list.
> 
> 
> ---------- Forwarded message ----------
> From: Ziliang Guo <ziliang@xxxxxxxxxxx>
> Date: Sat, Apr 7, 2012 at 12:11 PM
> Subject: RFC: Parallel classad evaluations
> To: Condor Flightworthy <condor-fw@xxxxxxxxxxx>
> 
> 
> As some of you may be aware of, I'm working on parallelizing classad
> matchmaking for my ME964 project.  Right now I have an implementation
> that takes in a single job classad and multiple machine ads and spins
> up multiple threads to search for a match amongst the machine ads.
> Another possibility is to have multiple threads each trying to find
> matches for a different job, though that one is a bit more complicated
> to pull off thanks to us modifying the machine ads during the
> matchmaking process (old classad compatibility hacks ftw....).  Does
> anyone else have any suggestions for other variations that may be
> useful or comments on the above two methods (keeping in mind only the
> single job vs multiple machine match has been implemented and tested
> so far)?

My immediate thoughts are that parallelism would be at the expense of
not being able to leverage the matchlist and that the schedd priorec
would possibly lead to serialization in the negotiator.

A possibility might be something based on matchlist where a new thread
is spawned on creation of matchlist. 

> 
> Long term there is also a question of whether there is any point in
> trying to parallelize the parsing of a classad from the raw text into
> its expression tree, or whether this is even possible.  Really long
> term, would it be possible to generate data structures such that we
> could attempt a match making operation on a GPU.
> 
> Note: Yes, I probably should have sent this out much earlier, like
> beginning of last week.
> 
> Z
> 
> --
> Condor Project Windows Developer
> 
>