[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[HTCondor-users] Slow schedd history queries in python



Hi,

This query:

condor_history -completedsince $(date -d "10 minutes ago" +"%sâ)

Takes on the order of a tenth of a second from the unix command line.

This bit of code:


shist_iter = schedd.history(
constraint='CompletionDate > %d' % (now - 600),
projection=[
"ProcId",
"ClusterId",
"JobStatus",
"AccountingGroup",
"AcctGroup",
"Owner",
"CompletionDate",
"CpusProvisioned",
"JobCurrentStartDate",
"JobCategory", "JobUniverse"
]
)

print("schedd hist query done")

# unfortunately, schedd.history returns a HistoryIterator, unlike Schedd.query which returns a list. The lines below
# make lists corresponding to the contents of the HistoryIterators, so we can use them more than once.

# unfortunately, schedd.history returns a HistoryIterator, unlike Schedd.query which returns a list. The lines below
# make lists corresponding to the contents of the HistoryIterators, so we can use them more than once.

print("start iter to list")

shist = list()
for ad in shist_iter:
shist.append(ad)

print("done iter to listâ)


Is similarly quick until the âfor ad in ââ bit, which takes order 7 seconds, even though there are only 160 records in that query. Is there a way to make this
similarly fast as the command line version?

JT



e