Hi Jacek,
I work as a Research Computing Facilitator in the Center
for High Throughput Computing, on campus, which produces the
HTCondor software.
You may get a more specific answer from the administrator
of the WEI's HTCondor pool, who will have configured the
specific policies for it. In the CHTC pool, for example, we
configure our execute servers (where jobs run) to allow jobs
to be pre-empted by higher-priority jobs after 72 hours. This
effectively establishes a 'cap' on how long jobs are
guaranteed to run (barring power/network interruptions, etc.).
Such runtime caps are implemented usually to balance factors
around slot turnover, fairness, and a desire to not run jobs
that are too long (because longer jobs are more susceptible to
other types of failures, and because computational work can
generally be run more efficiently by splitting a long job into
multiple shorter jobs).
There are other potential reasons for eviction, depending
on the specific configuration settings, such as interrupting
jobs that use far more memory or CPUs than the amount
requested. At some point, HTCondor will automatically evict in
such cases, purely to keep the execute server from crashing.
If you're not sure who the administrator of the cluster is,
let us know, and we can help make a connection since we're on
campus. Please otherwise let us know if we haven't helped you
get to an answer, or if anything I've described isn't very
clear.
Best,
Lauren Michael