Re: [HTCondor-devel] cgroups + SWAP

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 9/9/2013 9:22 AM, Joan J. Piles wrote:
> Hi Brian,
>
> Well my idea was more from the POV of a systems guy... Giving the choice
> to the users is (for me at least) optional and just a nicety... But I
> would like to limit the possibility of a user requesting 1Gb of RAM and
> then using over 10Gb, thus making the machine swap like hell and
> impacting the rest of users.
If user A has all of its processes in RAM and user B is swapping like 
hell, is user A actually impacted by user B ?  Or does user B only 
impact other users that have swapping processes?
> I know one option is limiting the swap, but
> what I'd rather have a big swap space just in case, and then limit the
> available swap to each job (proportionally to the RAM requested, for
> instance).
Limiting swap space makes sense in that swap is a shared resource and 
thus should be requested/managed, but is swap activity and swap size 
closely correlated?  Seems like what you are really worried about is 
lots of swap activity slowing down response time for all users of the 
system, not about exhaustion of swap space itself....
regards,
Todd

> And I think that making this tunable is just a small step worth it for
> the (admittedly few) users that would profit from this.
>
> Regards,
>
> Joan
>
> El 09/09/13 15:49, Brian Bockelman escribió:
> > On Sep 4, 2013, at 8:54 AM, Joan J. Piles <jpiles@xxxxxxxxx> wrote:
> >
> > > Hi all,
> > >
> > > We have recently started using cgroups to limit the RAM usage of our
> > > users. We want to avoid the situation where badly predicted
> > > requirements can bring down the whole machine where the job is
> > > executing, impacting other users' jobs.
> > >
> > > HTCondor does a great job using cgroups to achieve this, but I have
> > > the feeling that this can be improved.
> > >
> > > Right now, RAM usage is limited whilst swap is not. I am aware that
> > > you can tune this using swappiness parameters, but it is neither
> > > straightforward nor optimum, and furthermore it is something
> > > difficult to do on a per job basis.
> > >
> > > Right now HTCondor tunes the memory.limit_in_bytes or
> > > memory.soft_limit_in_bytes files within the cgroup to limit the RAM
> > > usage.
> > >
> > > I think HTCondor could provide a "request_swap" parameter in the
> > > submit file (and a RequestSwap associated job ClassAd) that whould be
> > > used to compute the value for memory.memsw.limit_in_bytes (which
> > > would of course be RequestMemory + RequestSwap).
> > >
> > > There would also be the associated MODIFY_REQUEST_EXPR_REQUESTSWAP
> > > which could be used (for instance) to limit the amount of swap
> > > reserved to a % of the RAM or to provide a sensible (or even
> > > unlimited) default.
> > >
> > > What do you think about this idea? I think it could easily piggyback
> > > on the existing cgroup infrastructure without too much hassle.
> > Hi Joan,
> >
> > I'm not too hot on this idea - how does the user know what value to
> > provide for RequestSwap?  Determining a working set size for an
> > application is a black art; knowing the memory requirements is hard
> > enough for most users!
> >
> > Brian
>
>
>
> _______________________________________________
> HTCondor-devel mailing list
> HTCondor-devel@xxxxxxxxxxx
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-devel

--
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                  Madison, WI 53706-1685