<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-GB" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I have noticed our site relies on the default value SelectTypeParameters (e.g. not specified in slurm.conf) which should be same as:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">SelectTypeParameters=CR_Core_Memory<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I have noticed some strange behaviour where the compute nodes do not seem to use memory cgroup e.g. in the log without the above specified in slurm.conf and assuming default value I see:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[2023-11-10T10:47:52.087] [7307368.batch] debug: task/cgroup: init: core enforcement enabled<o:p></o:p></p>
<p class="MsoNormal">[2023-11-10T10:47:52.088] [7307368.batch] debug: task/cgroup: init: device enforcement enabled<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">And with the above option in slurm.conf I see:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[2023-11-10T11:00:28.059] [7307370.extern] debug: task/cgroup: init: core enforcement enabled<o:p></o:p></p>
<p class="MsoNormal">[2023-11-10T11:00:28.059] [7307370.extern] debug: task/cgroup: task_cgroup_memory_init: task/cgroup/memory: TotCfgRealMem:191000M allowed:100%(enforced), swap:0%(enforced), max:100%(191000M) max+swap:100%(382000M) min:30M kmem:100%(191000M
permissive) min:30M<o:p></o:p></p>
<p class="MsoNormal">[2023-11-10T11:00:28.059] [7307370.extern] debug: task/cgroup: init: memory enforcement enabled<o:p></o:p></p>
<p class="MsoNormal">[2023-11-10T11:00:28.060] [7307370.extern] debug: task/cgroup: init: device enforcement enabled<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">This relates to <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><a href="https://github.com/SchedMD/slurm/blob/master/src/plugins/task/cgroup/task_cgroup.c">https://github.com/SchedMD/slurm/blob/master/src/plugins/task/cgroup/task_cgroup.c</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Specifically<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><a href="https://github.com/SchedMD/slurm/blob/d59bf870f660bed022f08aff073c9ae65119c37c/src/plugins/task/cgroup/task_cgroup.c#L78C2-L78C2">https://github.com/SchedMD/slurm/blob/d59bf870f660bed022f08aff073c9ae65119c37c/src/plugins/task/cgroup/task_cgroup.c#L78C2-L78C2</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Whilst I can only see the default possibly set on slurmctld but not on slurmd:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><a href="https://github.com/SchedMD/slurm/blob/d59bf870f660bed022f08aff073c9ae65119c37c/src/plugins/select/cons_tres/select_cons_tres.c#L288">https://github.com/SchedMD/slurm/blob/d59bf870f660bed022f08aff073c9ae65119c37c/src/plugins/select/cons_tres/select_cons_tres.c#L288</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Should the documentation be updated to require use of SelectTypeParameters or the code has better handling of defaulting this parameter?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Specifying the option seemed to fix odd behaviour but wanted to report the observation and whether I have misunderstood how this parameter is defaulted and whether other options also may need special care with defaults.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Tom<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas;color:black;mso-ligatures:none;mso-fareast-language:EN-GB">--</span><span style="color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas;color:black;mso-ligatures:none;mso-fareast-language:EN-GB">Thomas Green Senior Programmer</span><span style="color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas;color:black;mso-ligatures:none;mso-fareast-language:EN-GB">ARCCA, Redwood Building, King Edward VII Avenue, Cardiff, CF10 3NB</span><span style="color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas;color:black;mso-ligatures:none;mso-fareast-language:EN-GB">Tel: +44 (0)29 208 79269 Fax: +44 (0)29 208 70734</span><span style="color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas;color:black;mso-ligatures:none;mso-fareast-language:EN-GB">Email: <a href="mailto:greent10@cardiff.ac.uk"><span style="color:#000051">greent10@cardiff.ac.uk</span></a> Web: <a href="http://www.cardiff.ac.uk/arcca"><span style="color:#000051">http://www.cardiff.ac.uk/arcca</span></a></span><span style="color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas;color:black;mso-ligatures:none;mso-fareast-language:EN-GB"> </span><span style="color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas;color:black;mso-ligatures:none;mso-fareast-language:EN-GB">Thomas Green Uwch Raglennydd</span><span style="color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas;color:black;mso-ligatures:none;mso-fareast-language:EN-GB">ARCCA, Adeilad Redwood, King Edward VII Avenue, Caerdydd, CF10 3NB</span><span style="color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas;color:black;mso-ligatures:none;mso-fareast-language:EN-GB">Ffôn: +44 (0)29 208 79269 Ffacs: +44 (0)29 208 70734</span><span style="color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas;color:black;mso-ligatures:none;mso-fareast-language:EN-GB">E-bost: <a href="mailto:greent10@caerdydd.ac.uk"><span style="color:#000051">greent10@caerdydd.ac.uk</span></a> Gwefan: <a href="http://www.caerdydd.ac.uk/arcca"><span style="color:#000051">http://www.caerdydd.ac.uk/arcca</span></a></span><span style="color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-ligatures:none;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>