Re: [HTCondor-users] Concatenating history files?

["Pelletier, Michael V RTX" <Michael.V.Pelletier@xxxxxxx>]

Thanks very much for the tip, Cole!

 

My trouble is that we’re still on version 8, and since we’re drawing down the cluster in question there’s no funding to address an upgrade to version 10 or later. Sorry, I should have specified a version in my original message. Any alternatives available in v8? I’m thinking maybe not since the -search option may not have been introduced as a new feature.

 

A for loop with multiple invocations of condor_history -file should do the trick if that’s the only avenue available in the outdated release.

 

Michael Pelletier

Principal Technologist

High Performance Computing

Classified Infrastructure Services

 

C: +1 339.293.9149
michael.v.pelletier@xxxxxxx

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Cole Bollig via HTCondor-users
Sent: Wednesday, September 11, 2024 10:24 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Cc: Cole Bollig <cabollig@xxxxxxxx>
Subject: [External] Re: [HTCondor-users] Concatenating history files?

 

Hi Michael,

 

Since version V10.3.0, you can do condor_history -search /path/to/filename. This will find and read (in correct order) all matching timestamp rotated history files so in this example the following files would be parsed by condor_history:

• /path/to/filename

• /path/to/filename.20240911092145

• /path/to/filename.20240825155501

 

Cheers,

Cole Bollig

---

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Pelletier, Michael V RTX via HTCondor-users <htcondor-users@xxxxxxxxxxx>
Sent: Wednesday, September 11, 2024 9:10 AM
To: HTCondor-Users Mail List (htcondor-users@xxxxxxxxxxx) <htcondor-users@xxxxxxxxxxx>
Cc: Pelletier, Michael V RTX <Michael.V.Pelletier@xxxxxxx>
Subject: [HTCondor-users] Concatenating history files?

 

Hi folks,

 

I’ve got a huge amount of job history I’m trying to go through and summarize/categorize, to the tune of many gigabytes, and as you might expect it’s divided into a collection of rotated files with the usual timestamps.

 

I’m trying to use the -file option, so that it doesn’t bother the server and suffer the constraints of network connection and can work directly from a local filesystem where I’ve stashed the files.

 

Is there a way to enable condor_history to scan all the files in one fell swoop, rather than going through them one at a time with separate condor_history -file commands? I tried concatenating the files but it looks like the last line in each file has some metadata that condor_history pays attention to.

 

Thanks for any suggestions!

 

Michael V Pelletier

Principal Technologist

High Performance Computing

Classified Infrastructure Services