Re: [HTCondor-users] Concatenating history files?

[Cole Bollig <cabollig@xxxxxxxx>]

Hi Michael,

Since version V10.3.0, you can do condor_history -search /path/to/filename. This will find and read (in correct order) all matching timestamp rotated history files so in this example the following files would be parsed by condor_history:

• /path/to/filename
• /path/to/filename.20240911092145
• /path/to/filename.20240825155501

Cheers,

Cole Bollig

---

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Pelletier, Michael V RTX via HTCondor-users <htcondor-users@xxxxxxxxxxx>
Sent: Wednesday, September 11, 2024 9:10 AM
To: HTCondor-Users Mail List (htcondor-users@xxxxxxxxxxx) <htcondor-users@xxxxxxxxxxx>
Cc: Pelletier, Michael V RTX <Michael.V.Pelletier@xxxxxxx>
Subject: [HTCondor-users] Concatenating history files?

 

Hi folks,

 

I’ve got a huge amount of job history I’m trying to go through and summarize/categorize, to the tune of many gigabytes, and as you might expect it’s divided into a collection of rotated files with the usual timestamps.

 

I’m trying to use the -file option, so that it doesn’t bother the server and suffer the constraints of network connection and can work directly from a local filesystem where I’ve stashed the files.

 

Is there a way to enable condor_history to scan all the files in one fell swoop, rather than going through them one at a time with separate condor_history -file commands? I tried concatenating the files but it looks like the last line in each file has some metadata that condor_history pays attention to.

 

Thanks for any suggestions!

 

Michael V Pelletier

Principal Technologist

High Performance Computing

Classified Infrastructure Services