[HTCondor-users] How to use a function in a constraint expression in condor_history and.python bindings

[Joseph Areeda <newsreply@xxxxxxxxxx>]

I know there is a simple answer but it eludes me.

What I hope to do:

We are about to implement cgroups to manage resources for local universe jobs. Iâd like to grep the condor history for reports of the disk and memory usage of the affected jobs.

The program condor_history will get us started but a python program has long term uses.

What Iâve tried:

CLI:

$ condor_history -constraint 'regex(".*gwsum.*", Arguments)' -limit 5  -af:jht OsUser ResidentSetSize_RAW RequestMemory RequestDisk RequestCpus  JobCurrentStartDate Cmd Arguments| grep -v /usr/bin/condor_dagman

Python:

import htcondor

import lassa

import classad

schedd = htcondor.Schedd()

projection = ['OsUser', 'ResidentSetSize_RAW', 'RequestMemory', 'RequestDisk', 'RequestCpus', 'JobCurrentStartDate', 'Cmd', 'Arguments']

constraint = 'regex(".*gwsumm.*", Arguments)â

match=2

for ad in schedd.history(constraint= constraint, projection=projection, match=match):

    print(ad)

Problems:

• The CLI example does not match any jobs but if use simple string compare it works:

$ condor_history -constraint 'OsUser=="detchar"' -limit 2  -af:jht OsUser ResidentSetSize_RAW RequestMemory RequestDisk RequestCpus  JobCurrentStartDate Cmd Arguments| grep -v /usr/bin/condor_dagman | cut -c 1-140

1: ID           OsUser  ResidentSetSize_RAW     RequestMemory   RequestDisk     RequestCpus     JobCurrentStartDate     Cmd     Arguments

2:9444416.0     detchar 5970092                 4096            5242880         1               1755114925              /home/detchar/.conda/envs/ligo-summary-3

3:9444454.0     detchar 2416684                 65536           16777216        1               1755114959              /home/detchar/.conda/envs/ligo-summary-3

• The python code above churns for a while then produces:

HTCondorIOError                           Traceback (most recent call last)

Cell In[13], line 1

----> 1 for ad in schedd.history(constraint= constraint, projection=projection, match=match):

      2     print(ad)

File ~/.conda/envs/ligo-omicron-3.10-test/lib/python3.10/site-packages/htcondor/_lock.py:70, in add_lock.<locals>.wrapper(*args, **kwargs)

     67 try:

     68     acquired = LOCK.acquire()

---> 70     rv = func(*args, **kwargs)

     72     # if the function returned a context manager,

     73     # create a LockedContext to manage the lock

     74     is_cm = is_context_manager(rv)

HTCondorIOError: Timeout when waiting for remote host

• Using the simple string compared works as expected.

Long winded way o say I donât know how to use a function in a constraint _expression_.

Any help will be appreciated.

Joe