Re: [HTCondor-users] memory leak in htcondor2 job event log?

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 12/19/2025 10:17 AM, David Schultz wrote:

Hi,

I switched from htcondor to htcondor2 two weeks ago, and have noticed significantly higher memory usage (orders of magnitude) when using job event logs, which always increases until I restart my python script.

So I wrote a test script [1] and ran it with both the old htcondor pybinding and the new htcondor2 pybinding.  The old version maintains a consistent value in memory, while the htcondor2 version always increases in memory.

Can someone from the HTCondor team take a look at the job event log code and see if there's a bug in there leaking memory?

    Hi David,
    
    We absolutely will, stay tuned!  
    
    Thank you for such a helpful and clear bug report!
    
    regards,
    Todd

Best,

David

[1] test_job_event_log.py

import tracemalloc
#import htcondor
import htcondor2 as htcondor

def get_memory_from_proc_statm():
    with open('/proc/self/statm') as f:
        # The second value is the Resident Set Size (RSS) in pages
        pages = int(f.read().split()[1])
        page_size = 4096
        rss_bytes = pages * page_size
        return rss_bytes

def mem_report():
  current, _ = tracemalloc.get_traced_memory()
  print(f"Python memory usage: {current / 1024:.2f} KB")
  print(f"Proc memory usage: {get_memory_from_proc_statm() / 1024:.2f} KB")

# use a job event log that already exists somewhere (this one was 100MB)

jel_filename = 'submit/2025-12-19T13/jobs.jel'

def run():
  jel = htcondor.JobEventLog(jel_filename)
  events = jel.events(0)

  print('start')
  mem_report()

  while True:
    i = 0
    for i,_ in enumerate(events):
      if i >= 10000:
        break
    print(f'loaded {i} events')
    mem_report()
    if i == 0:
      break

  jel.close()
  print('close log')
  mem_report()

  print('end')
  mem_report()

tracemalloc.start()
run()
tracemalloc.stop()

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe

The archives can be found at: https://www-auth.cs.wisc.edu/lists/htcondor-users/ 

-- 
Todd Tannenbaum <tannenba@xxxxxxxxxxx>  University of Wisconsin-Madison
Center for High Throughput Computing    Department of Computer Sciences
Calendar: https://tinyurl.com/yd55mtgd  1205 University Ave.
Phone: (608) 263-7132                   Madison, WI 53706
Personal Zoom Room: https://uwmadison.zoom.us/my/tannenba