Re: [HTCondor-devel] Examining in-memory representation of ClassAds

[Brian Bockelman <bbockelm@xxxxxxxxxxx>]

On May 28, 2013, at 10:00 AM, Greg Thain <gthain@xxxxxxxxxxx> wrote:

> On 05/28/2013 08:04 AM, Brian Bockelman wrote:
>> With the in-memory format changes described above, memory usage for the 22MB sample corpus is 29MB.  While the savings aren't as significant as I had hoped (wouldn't it be nice to have the text format be larger than the parsed form?), it's a 77% savings over current with compression and 82% savings over current without compression.
> 
> As both the schedd and the shadow really only need to evaluate a small fraction of the expressions in any given ad, one possibility is to build the parse tree only on demand, and keep the unparsed form around otherwise.  (We'd still need to be able to validate the textual form, but presumably that's pretty lightweight, compared to building and copying all the expr trees).
> 

That's basically the approach here - the packed format is basically a much-easier-to-rebuild binary format for ClassAds.  Note in my description neither the ClassAd nor the cache actually contains an ExprTree object.  This comes at a cost - see below.

For example, an expression containing only a string literal 'foo' would have an encoding:

(header byte) (integer encoding for '3') (char array of length 3)

0x81 0x01 0x03 f o o

(6 bytes) Compared to having the encoding of:

"foo"

(6 bytes using a nul-terminated scheme) it's far less human readable but much faster to unpack into an expression tree.

As I mentioned, you now have a lifetime issue - you create the "ExprTree *" on-demand, but when do you delete it?  That leads to using shared_ptr<ExprTree>, which then leads to changing many interfaces (which should be done anyway to clean up ambiguities in pointer ownership!).  The in-memory savings are large, but at the cost of changing/fixing interfaces.

You're also going to take a hit in CPU performance due to many more heap allocations, but I suspect that's not a current bottleneck.  There's some work that could be done to ameliorate this hit.  For example, code doesn't seem to touch Lookup directly but rather uses "EvaluateAttrInt" or equivalent.  In such a case, you could have a fast-path optimization for packed literals; you can create the integer directly from the packed string without creating an ExprTree.  If there is an actual expression to evaluate, the overhead of the malloc is likely tolerable compared.

Brian

Attachment: smime.p7s
Description: S/MIME cryptographic signature