<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Dear all,<br>
<br>
I must say, I'm a bit dazzled, since this configuration should not
be valid. This is something, I myself observed. According to the
manpage of slurm.conf, CPUs and Boards are mutually exclusive:<br>
<br>
*Boards* Number of Baseboards in nodes with a baseboard
controller. Note that when Boards is specified, SocketsPerBoard,
CoresPerSocket, and ThreadsPerCore should be specified. Boards and
CPUs are mutually exclusive. The default value is 1.<br>
<br>
<br>
Since I needed to tell slurm to not use the hyperthreads, I halved
the CPUs value and omitted Boards. This configuration works for me:<br>
<br>
NodeName=linuxbmc[0021-0036] CPUs=12 SocketsPerBoard=2
CoresPerSocket=6 ThreadsPerCore=2 RealMemory=96508 TmpDisk=41035
State=UNKNOWN<br>
<br>
Yet I do not know, why these options are mutually exclusive, since
from my sight of view CPUs is the number of cores/threads, I want
slurm to schedule. The other values, Boards, SocketsPerBoard,
CoresPerSocket and ThreadsPerCore are "architectural" values, which
should help in placing the tasks on the node itself, compare --hint
of sbatch.<br>
<br>
<br>
Best<br>
Marcus<br>
<br>
<br>
<div class="moz-cite-prefix">On 05/03/2018 02:28 AM, Matt Hohmeister
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:DM5PR02MB3179269878ED8E1BCF4149228A870@DM5PR02MB3179.namprd02.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">I have a two-node cluster: the
server/compute node is a Dell PowerEdge R730; the compute
node, a Dell PowerEdge R630. On both of these nodes,
<span style="font-family:Consolas">slurmd -C</span> gives me
the exact same line:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:Consolas">[me@odin
slurm]$ slurmd -C<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas">NodeName=odin
CPUs=40 Boards=1 SocketsPerBoard=2 CoresPerSocket=10
ThreadsPerCore=2 RealMemory=128655<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas">[me@thor
slurm]$ slurmd -C<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas">NodeName=thor
CPUs=40 Boards=1 SocketsPerBoard=2 CoresPerSocket=10
ThreadsPerCore=2 RealMemory=128655<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">So I edited my slurm.conf appropriately:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:Consolas">NodeName=odin
CPUs=40 Boards=1 SocketsPerBoard=2 CoresPerSocket=10
ThreadsPerCore=2 RealMemory=128655<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas">NodeName=thor
CPUs=40 Boards=1 SocketsPerBoard=2 CoresPerSocket=10
ThreadsPerCore=2 RealMemory=128655<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">…and it looks good, except for the drain on
my server/compute node:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:Consolas">[me@odin
slurm]$ sinfo<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas">PARTITION
AVAIL TIMELIMIT NODES STATE NODELIST<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas">debug*
up infinite 1 drain odin<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas">debug*
up infinite 1 idle thor
<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">…for the following reason:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:Consolas">[me@odin
slurm]$ sinfo -R<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas">REASON
USER TIMESTAMP NODELIST<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Consolas">Low
socket*core*thre slurm 2018-05-02T11:55:38 odin<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Any ideas?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks!<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Matt Hohmeister<o:p></o:p></p>
<p class="MsoNormal">Systems and Network Administrator<o:p></o:p></p>
<p class="MsoNormal">Department of Psychology<o:p></o:p></p>
<p class="MsoNormal">Florida State University<o:p></o:p></p>
<p class="MsoNormal">PO Box 3064301<o:p></o:p></p>
<p class="MsoNormal">Tallahassee, FL 32306-4301<o:p></o:p></p>
<p class="MsoNormal">Phone: +1 850 645 1902<o:p></o:p></p>
<p class="MsoNormal">Fax: +1 850 644 7739<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Marcus Wagner, Dipl.-Inf.
IT Center
Abteilung: Systeme und Betrieb
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Tel: +49 241 80-24383
Fax: +49 241 80-624383
<a class="moz-txt-link-abbreviated" href="mailto:wagner@itc.rwth-aachen.de">wagner@itc.rwth-aachen.de</a>
<a class="moz-txt-link-abbreviated" href="http://www.itc.rwth-aachen.de">www.itc.rwth-aachen.de</a>
</pre>
</body>
</html>