Re: [HTCondor-users] memory leak in condor_q?

[Brian Bockelman <bbockelm@xxxxxxxxxxx>]

Hi Daniel,

This is expected.  Stream does not stream (well, until my patches land).  It still buffers the entire response in memory before parsing it.

Stream prevents HTCondor from sorting the results in memory.

Non-blocking condor_q patch set will take care of this and turn it into a real stream.

Sent from my iPhone

> On Feb 6, 2014, at 9:46 AM, Pek Daniel <pekdaniel@xxxxxxxxx> wrote:
> 
> I've noticed there's not so much difference between the memory
> consumption of condor_q -stream and condor_q:
> 
> [root@XXX thrash]# /usr/bin/time -v condor_q -stream >/dev/null
> ...
> Maximum resident set size (kbytes): 186592
> ...
> 
> [root@XXX thrash]# /usr/bin/time -v condor_q >/dev/null
> ...
> Maximum resident set size (kbytes): 306352
> ...
> 
> There was 100k jobs in the queue. I've dug into the source a bit, and
> I suspect some leak somewhere here:
> https://github.com/htcondor/htcondor/blob/b151357dcd13efe2703a2386e1d89bbacac79cd6/src/condor_schedd.V6/qmgmt_send_stubs.cpp#L862-L882
> 
> or here:
> https://github.com/htcondor/htcondor/blob/0222c71b4a7cf5946ab9d5caf5ecca0ca8c75539/src/condor_utils/classad_oldnew.cpp#L57-L130
> 
> Maybe the ReliSock, or the ClassAd...
> 
> Cheers,
> daniel
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/