Re: [HTCondor-users] Fwd: Re: Docker universe ImageSize/MemoryUsage

[Greg Thain <gthain@xxxxxxxxxxx>]

On 08/07/2017 10:11 PM, Sitharaman, Harish Mahadevan wrote:

So this error seems intermittent (maybe multiple jobs on a single machine? boto3 multi-part upload?). I've tried executing this job as a bare-bone docker run and the MEM USAGE "docker stats" seems reasonably within the LIMIT and the job finished successfully.

Also,

1. From the job.log, the HUGE MemoryUsage is reported immediately after the job starts executing on the worker machine. Does that have anything to do with how condor_starter wraps docker?

It could -- you can look in the StarterLog.slotXXX and see the exact docker run arguments that condor uses to start the job.

2. Shouldn't condor ideally have complained/HELD with "Docker job has gone over the memory limit" log?

Yes, but it relies on docker setting up the cgroups properly do to this.

-greg