[slurm-users] ConstrainRAMSpace=yes and page cache?
juergen.salk at uni-ulm.de
Fri Jun 21 21:44:49 UTC 2019
* Christopher Samuel <chris at csamuel.org> [190621 09:59]:
> On 6/13/19 5:27 PM, Kilian Cavalotti wrote:
> > I would take a look at the various *KmemSpace options in cgroups.conf,
> > they can certainly help with this.
> Specifically I think you'll want:
> to fix this. This happens for NFS and Lustre based systems, I don't think
> it's a problem for GPFS as mmfsd has its own pagepool separate to the
> processes address space.
ConstrainKmemSpace=no seems to be the default anyway, but I've just
tried with having ConstrainRAMSpace=yes and ConstrainKmemSpace=no
explicitly set in cgroups.conf (and without
JobAcctGatherParams=OverMemoryKill in slurm.conf).
However this apparently still caps the amount of memory available for
page cache to the amount of memory requested at job submission (by
means of --mem=.. option) minus the RSS portion occupied by the
processes of the job. This leaves any other unused memory aside that
could have actually been efficiently used for caching file operations.
I've checked this for I/O on locally attached block storage though. No
NFS, no Lustre.
Scientific Software & Compute Services (SSCS)
Kommunikations- und Informationszentrum (kiz)
Telefon: +49 (0)731 50-22478
Telefax: +49 (0)731 50-22471
More information about the slurm-users