[slurm-users] ConstrainRAMSpace=yes and page cache?

[Juergen Salk]

* Christopher Samuel <chris at csamuel.org> [190621 09:59]:
> On 6/13/19 5:27 PM, Kilian Cavalotti wrote:
> 
> > I would take a look at the various *KmemSpace options in cgroups.conf,
> > they can certainly help with this.
> 
> Specifically I think you'll want:
> 
> ConstrainKmemSpace=no
> 
> to fix this.  This happens for NFS and Lustre based systems, I don't think
> it's a problem for GPFS as mmfsd has its own pagepool separate to the
> processes address space.


Hi Chris,

ConstrainKmemSpace=no seems to be the default anyway, but I've just 
tried with having ConstrainRAMSpace=yes and ConstrainKmemSpace=no
explicitly set in cgroups.conf (and without
JobAcctGatherParams=OverMemoryKill in slurm.conf). 

However this apparently still caps the amount of memory available for
page cache to the amount of memory requested at job submission (by
means of --mem=.. option) minus the RSS portion occupied by the
processes of the job. This leaves any other unused memory aside that
could have actually been efficiently used for caching file operations. 

I've checked this for I/O on locally attached block storage though. No
NFS, no Lustre.

Best regards
Jürgen

-- 
Jürgen Salk
Scientific Software & Compute Services (SSCS)
Kommunikations- und Informationszentrum (kiz)
Universität Ulm
Telefon: +49 (0)731 50-22478
Telefax: +49 (0)731 50-22471