To get similar performance, you need to use the since argument of python history rather than (or in addition to) the
constraint argument.
constraint selects which records to return
since controls when to stop scanning the history file.
-tj
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Jeff Templon <templon@xxxxxxxxx>
Sent: Thursday, May 22, 2025 8:39 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] Slow schedd history queries in python
Addendum:
Is there a way to make this
similarly fast as the command line version?
should have made explicit that I meant, can the schedd.history() usage be made as fast as the command line version.
Of course, running the unix command under the hood is always a possibility.
JT
On 22 May 2025, at 15:30, Jeff Templon <templon@xxxxxxxxx> wrote:
Hi,
This query:
condor_history -completedsince $(date -d "10 minutes ago" +"%s”)
Takes on the order of a tenth of a second from the unix command line.
This bit of code:
shist_iter
= schedd.history(
constraint='CompletionDate >
%d'
% (now
-
600),
projection=[
"ProcId",
"ClusterId",
"JobStatus",
"AccountingGroup",
"AcctGroup",
"Owner",
"CompletionDate",
"CpusProvisioned",
"JobCurrentStartDate",
"JobCategory",
"JobUniverse"
]
)
print("schedd hist query done")
# unfortunately, schedd.history returns a HistoryIterator, unlike Schedd.query which returns a list. The lines below
# make lists corresponding to the contents of the HistoryIterators, so we can use them more than once.
# unfortunately, schedd.history returns a HistoryIterator, unlike Schedd.query which returns a list. The lines below
# make lists corresponding to the contents of the HistoryIterators, so we can use them more than once.
print("start iter to list")
shist
= list()
for
ad in
shist_iter:
shist.append(ad)
print("done iter to list”)
Is similarly quick until the “for ad in …” bit, which takes order 7 seconds, even though there are only 160 records in that query. Is there a way to make this
similarly fast as the command line version?
JT
e
|