Re: [Condor-devel] RFC: Parallel classad evaluations

[Will Benton <willb@xxxxxxxxxx>]

----- Original Message -----
> From: "Brian Bockelman" <bbockelm@xxxxxxxxxxx>
> To: "Matthew Farrellee" <matt@xxxxxxxxxx>
> Cc: "Condor Developers" <condor-devel@xxxxxxxxxxx>
> Sent: Tuesday, April 10, 2012 7:51:25 AM
> Subject: Re: [Condor-devel] RFC: Parallel classad evaluations
> 
> 

[context trimmed]

> I happened to be poking around with a memory profiler yesterday.  It
> seems that memory allocations are one of the bottlenecks in the
> system.
> 
> I would seriously consider looking into trying a pool-based allocator
> for classads - especially the lexer.  This gives you constant-time
> allocation.  My understanding is that GCC switched to obstacks /
> pool-based allocators for some of their time-critical pieces.

+1 to the general idea of looking at better allocators.  Z, I have some (research) expertise in this area; if you're interested, I can point you to some things worth looking at the next time I'm at UW.

(More generally, I'd say that in a vast preponderance of the cases where someone has suggested that I should investigate a difficult parallelization effort in some preexisting code, I've found serial code improvements that have resulted in better performance than we expected from the parallel implementation.  This observation probably doesn't do you a lot of good if the point of your class project is to parallelize something, but it might suggest some broadly-applicable insights.)


best,
wb