Re: [HTCondor-users] Slow schedd history queries in python

[Jeff Templon <templon@xxxxxxxxx>]

Addendum:

Is there a way to make this

similarly fast as the command line version?

should have made explicit that I meant, can the schedd.history() usage be made as fast as the command line version.

Of course, running the unix command under the hood is always a possibility.

JT

On 22 May 2025, at 15:30, Jeff Templon <templon@xxxxxxxxx> wrote:

Hi,

This query:

condor_history -completedsince $(date -d "10 minutes ago" +"%sâ)

Takes on the order of a tenth of a second from the unix command line.

This bit of code:

shist_iter = schedd.history(

constraint='CompletionDate > %d' % (now - 600),

projection=[

"ProcId",

"ClusterId",

"JobStatus",

"AccountingGroup",

"AcctGroup",

"Owner",

"CompletionDate",

"CpusProvisioned",

"JobCurrentStartDate",

"JobCategory", "JobUniverse"

]

)

print("schedd hist query done")

# unfortunately, schedd.history returns a HistoryIterator, unlike Schedd.query which returns a list. The lines below

# make lists corresponding to the contents of the HistoryIterators, so we can use them more than once.

# unfortunately, schedd.history returns a HistoryIterator, unlike Schedd.query which returns a list. The lines below

# make lists corresponding to the contents of the HistoryIterators, so we can use them more than once.

print("start iter to list")

shist = list()

for ad in shist_iter:

shist.append(ad)

print("done iter to listâ)

Is similarly quick until the âfor ad in ââ bit, which takes order 7 seconds, even though there are only 160 records in that query. Is there a way to make this

similarly fast as the command line version?

JT

e

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe

Join us in June at Throughput Computing 25: https://osg-htc.org/htc25

The archives can be found at: https://www-auth.cs.wisc.edu/lists/htcondor-users/