Re: [HTCondor-users] python multiple job event logs in parallel

[Cole Bollig <cabollig@xxxxxxxx>]

Hi David,

Depending on what the code is doing, it may be beneficial to turn on the Global event log, and just read all events from the single file. This could be useful for a program that monitors the entire AP since the global event log records all events for all jobs in a single file.

-Cole Bollig

---

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of David Schultz <david.schultz@xxxxxxxxxxxxxxxx>
Sent: Thursday, October 5, 2023 4:00 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] python multiple job event logs in parallel

 

Hi,

I'm looking at migrating code from querying the schedd directly to reading job event logs, but I would need to query multiple log files.  The current python bindings make this a sequential operation, even trying from multiple threads, due to a global mutex.

So, requests:

1. If possible, could this mutex be per-log?

2. Allow polling of the job event log (a true 0 timeout). This would allow checking multiple logs for updates sequentially much faster.  Since I can see that this might break existing code, allowing a float for a 1 millisecond timeout would also work well, as it seems like the C++ code natively uses milliseconds.

Best,

David