[HTCondor-users] Running System Tests on HTCondor

[Dudu Handelman <duduhandelman@xxxxxxxxxxx>]

Hi all,

I’m looking for some input on how we might approach this.

Our developers run extensive system and performance tests as part of the regular development pipeline. Currently, system tests are executed on virtual machines and multiple Docker containers, while performance tests are performed similarly but on bare-metal machines.

These tests rely on a stack composed of multiple servers and containers.

For example:

A typical test stack consists of four servers or containers: three Kafka instances and one container running the application. This application reads and writes to Kafka and verifies data integrity.

I’d like to explore the possibility of running the system tests on HTCondor. This would significantly reduce the overall testing time, and we already have the necessary capacity available.

One option is to package all services into a single Docker container, but I’d prefer to avoid this approach if possible.

Maybe docker in docker

Thanks in advance for any ideas or suggestions.

David