<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>We've been typically taking 4G off the top for memory in our
slurm.conf for the system and other processes. This seems to work
pretty well.</p>
<p>-Paul Edmon-<br>
</p>
<br>
<div class="moz-cite-prefix">On 01/17/2018 01:44 AM, Marcin Stolarek
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAC8K6BN8n7pKJzSt+E=LE0MKdXZmiHfO_QmW2SaGPDQiKWD-AA@mail.gmail.com">
<div dir="ltr">
<div>I think that it depends on your kernel and the way the
cluster is booted (for instance initrd size). You can check
the memory used by kernel in dmesg output - search for the
line starting with "Memory:". This is fixed. <br>
</div>
<div>It may be also good idea to "reserve" some space for cache
and buffers - check htop or /proc/meminfo (Slab) this may
depend on your OS (filesystem, hardware modules) and if you
have a limited set of applications - workload. Size of this
part of memory may depend on "node size", number of cores
should be good measurement. <br>
</div>
<div><br>
</div>
<div>cheers,</div>
<div>Marcin<br>
</div>
<div>
<div><br>
</div>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">2018-01-17 6:03 GMT+01:00 Greg Wickham
<span dir="ltr"><<a href="mailto:greg.wickham@kaust.edu.sa"
target="_blank" moz-do-not-send="true">greg.wickham@kaust.edu.sa</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
We’re using cgroups to limit memory of jobs, but in our
slurm.conf the total node memory capacity is currently
specified.<br>
<br>
Doing this there could be times when physical memory is over
subscribed (physical allocation per job plus kernel memory
requirements) and then swapping will occur.<br>
<br>
Is there a recommended “kernel overhead” memory (either % or
absolute value) that we should deduct from the total
physical memory?<br>
<br>
thanks,<br>
<br>
-greg<br>
<br>
--<br>
Dr. Greg Wickham<br>
Advanced Computing Infrastructure Team Lead<br>
Advanced Computing Core Laboratory<br>
King Abdullah University of Science and Technology<br>
Building #1, Office #0124<br>
<a href="mailto:greg.wickham@kaust.edu.sa"
moz-do-not-send="true">greg.wickham@kaust.edu.sa</a> <a
href="tel:%2B966%20544%20700%20330" value="+966544700330"
moz-do-not-send="true">+966 544 700 330</a><br>
--<br>
<br>
<br>
______________________________<wbr>__<br>
This message and its contents including attachments are
intended solely for the original recipient. If you are not
the intended recipient or have received this message in
error, please notify me immediately and delete this message
from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment
before printing this email.<br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>