Re: [HTCondor-users] Thin LVM over commit how to?

[Cole Bollig <cabollig@xxxxxxxx>]

Hi Alexandr,

I did some testing. You can set the DISK configuration option to have a larger value than the backing thinpool allowing for more matches or larger jobs. However, when DISK is explicitly set, we don't query the backing LVM store for the available size. So, doing a programmatic definition of DISK = DISK * 1.2 is using another detected disk as the base for the operation (I believe it is the size of the filesystem EXECUTE is located on). I was able to set DISK to 10GB with a backing thinpool of 8GB and see a sleep job requesting 9GB run.

That being said, with thin provisioning, by default, the ephemeral logical volumes have an extra 2GB added to their size by Condor. So, if the majority of the jobs are requesting up to 1GB of disk then I would expect only up to 1/3 of the volume to be utilized most of the time. Which is fine since the disk is only allocated as it is used.

Cheers,

Cole Bollig

---

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Alexandr Mikula <mikula@xxxxxx>
Sent: Tuesday, October 14, 2025 8:43 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] Thin LVM over commit how to?

 

On Tue, 2025-10-14 at 13:37 +0000, Cole Bollig via HTCondor-users
wrote:
>
> Hi Alexandr,
>
>
> What exactly are you trying to overcommit? The backing thinpool LV or
> the HTCondor ephemeral thin LVs produced for each job?
>
>
> Cheers,
> Cole Bollig
Hi Cole,
I guess I want to over commit the backing thinpool.
I want to fit in more lvs than the size of the thin pool since this is
one of the features of the thin pools.
We see usually less than 20% usage on our ephemeral volumes, so it
makes sense to bit over commit the space.
We see the lvms as a way to handle stray output jobs with a benefit of
the isolation.

Cheers
AM
--
Alexandr Mikula
Oddělení síťování a výpočetní techniky & Výpočetní středisko
Fyzikální ústav Akademie věd České republiky, v. v. i.
Institute of Physics of the Czech Academy of Sciences