<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hi All,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I'm hoping that someone many have encountered this scenario and knows of a solution. Basically we wish to change the default core distribution but only for specific compute nodes. The current default distribution is cyclic, but for specific
nodes we would like to override this behaviour to be a default block:block.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We're currently using Slurm 17.11.12. I'm aware that --distribution (-m) can set at a job level but this relies on the user setting it and I would prefer not to be dependent on that. I'm also aware that I can set CR_CORE_DEFAULT_DIST_BLOCK
via slurm.conf but this is a global change. I'm not aware of a node specific setting.<o:p></o:p></p>
<p class="MsoNormal"><o:p></o:p></p>
<p class="MsoNormal">I've tried a couple of methods using SPANK and LUA (LUA would be our preferred mechanism). With both SPANK and LUA I'm able to set SBATCH_DISTRIBUTION to block:block. With LUA I'm setting the SBATCH_DISTRIBUTION via the job_desc environment
variable, as SBATCH_DISTRIBUTION doesn't appear to be directly available via job_desc.distribution. With SPANK the SLURM_DISTRIBUTION output variable doesn't appear to get set, whilst with LUA I'm able to see the SLURM_DISTRIBUTION getting populated with block:block.
But even though SLURM_DISTRIBUTION is set the scheduler doesn't appear to honour the request as I still end up a cyclic distribution.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Has anyone encountered this before? Can anyone suggest an alternative solution to achieve the end?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b><span style="color:black;mso-fareast-language:EN-GB">---<o:p></o:p></span></b></p>
<p class="MsoNormal"><b><span style="color:black;mso-fareast-language:EN-GB">Samuel Gallop<o:p></o:p></span></b></p>
<p class="MsoNormal"><i><span style="font-size:10.0pt;color:black;mso-fareast-language:EN-GB">Computing infrastructure for Science<o:p></o:p></span></i></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;color:black;mso-fareast-language:EN-GB">CiS Support & Development</span></b><b><span style="font-size:6.0pt;color:black;mso-fareast-language:EN-GB"><o:p></o:p></span></b></p>
<p class="MsoNormal"><b><span style="font-size:6.0pt;color:black;mso-fareast-language:EN-GB"><o:p> </o:p></span></b></p>
</div>
</body>
</html>