<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Hi Chris,
<div class=""><br class="">
</div>
<div class="">Working on weekends - hey ?</div>
<div class=""><br class="">
</div>
<div class="">when i do "slurmd -C” on one of my execute node, i get: </div>
<div class=""><br class="">
</div>
<div class="">
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class=""><b class="">eric@radonc01</b>:<b class="">~</b>$ slurmd -C</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">NodeName=radonc01 slurmd: Considering each NUMA node as a socket</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">CPUs=32 Boards=1 SocketsPerBoard=4 CoresPerSocket=8 ThreadsPerCore=1 RealMemory=64402</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">UpTime=2-17:35:12</span></div>
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Also, when i do “lscpu” i get:</div>
<div class=""><br class="">
</div>
<div class="">
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class=""><b class="">eric@radonc01</b>:<b class="">~</b>$ lscpu</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">Architecture: x86_64</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">CPU op-mode(s): 32-bit, 64-bit</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">Byte Order: Little Endian</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">CPU(s): 32</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">On-line CPU(s) list: 0-31</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">Thread(s) per core: 2</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">Core(s) per socket: 8</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">Socket(s): 2</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">NUMA node(s): 4</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">Vendor ID: AuthenticAMD</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">CPU family: 21</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">Model: 2</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">Model name: AMD Opteron(tm) Processor 6376</span></div>
<div style="margin: 0px; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">Stepping: 0</span></div>
</div>
<div class="">“”</div>
<div class="">“”</div>
<div class="">“”</div>
<div class="">“”</div>
<div class="">“”</div>
<div class=""><br class="">
</div>
<div class="">
<div style="margin: 0px; font-size: 11px; line-height: normal; background-color: rgb(255, 255, 255);" class="">
It seems as the commands give different result (?) - What do you think ?</div>
</div>
<div style="margin: 0px; font-size: 11px; line-height: normal; background-color: rgb(255, 255, 255);" class="">
<br class="">
</div>
<div style="margin: 0px; font-size: 11px; line-height: normal; background-color: rgb(255, 255, 255);" class="">
<br class="">
</div>
<div class="">
<div class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div style="text-align: -webkit-auto; orphans: 2; widows: 2; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div style="orphans: auto; widows: auto;" class=""><span style="text-align: -webkit-auto; background-color: rgba(255, 255, 255, 0);" class="">_____________________________________________________________________________________________________</span></div>
<div style="orphans: auto; widows: auto;" class=""><span style="background-color: rgba(255, 255, 255, 0);" class=""><br class="">
</span></div>
<span style="background-color: rgba(255, 255, 255, 0);" class=""><b class="">
<div style="orphans: auto; widows: auto;" class=""><b style="text-align: -webkit-auto;" class="">Eric F. Alemany</b></div>
</b>
<div style="orphans: auto; widows: auto;" class=""><i style="text-align: -webkit-auto;" class="">System Administrator for Research</i></div>
</span>
<div style="orphans: auto; widows: auto;" class=""><span style="background-color: rgba(255, 255, 255, 0);" class=""><br class="">
</span></div>
<div style="orphans: auto; widows: auto;" class=""><span style="text-align: -webkit-auto; background-color: rgba(255, 255, 255, 0);" class="">Division of Radiation & Cancer Biology</span></div>
<div style="orphans: auto; widows: auto;" class=""><span style="text-align: -webkit-auto; background-color: rgba(255, 255, 255, 0);" class="">Department of Radiation Oncology</span></div>
<div style="orphans: auto; widows: auto;" class=""><span style="background-color: rgba(255, 255, 255, 0);" class=""><br class="">
</span></div>
<div style="orphans: auto; widows: auto;" class=""><span style="text-align: -webkit-auto; background-color: rgba(255, 255, 255, 0);" class="">Stanford University School of Medicine</span></div>
<div style="orphans: auto; widows: auto;" class=""><span style="text-align: -webkit-auto; background-color: rgba(255, 255, 255, 0);" class="">Stanford, California 94305</span></div>
<div style="orphans: auto; widows: auto;" class=""><span style="background-color: rgba(255, 255, 255, 0);" class=""><br class="">
</span></div>
<div style="orphans: auto; widows: auto;" class=""><span style="background-color: rgba(255, 255, 255, 0);" class=""><font style="text-align: -webkit-auto;" class="">Tel:</font><a href="tel:1-650-498-7969" x-apple-data-detectors="true" x-apple-data-detectors-type="telephone" x-apple-data-detectors-result="1" style="text-align: -webkit-auto;" class="">1-650-498-7969</a><font style="text-align: -webkit-auto;" class="">
No Texting</font></span></div>
<div style="orphans: auto; widows: auto;" class=""><span style="background-color: rgba(255, 255, 255, 0);" class=""><font style="text-align: -webkit-auto;" class="">Fax:</font><a href="tel:1-650-723-7382" x-apple-data-detectors="true" x-apple-data-detectors-type="telephone" x-apple-data-detectors-result="2" style="text-align: -webkit-auto;" class="">1-650-723-7382</a></span></div>
<div style="orphans: auto; widows: auto;" class=""><br class="">
</div>
</div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<font style="background-color: rgba(255, 255, 255, 0);" class=""></font></div>
</div>
</div>
<br class="Apple-interchange-newline">
</div>
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On May 5, 2018, at 5:42 AM, Chris Samuel <<a href="mailto:chris@csamuel.org" class="">chris@csamuel.org</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">On Saturday, 5 May 2018 2:45:19 AM AEST Eric F. Alemany wrote:<br class="">
<br class="">
<blockquote type="cite" class="">With Ray suggestion i have a error message for each nodes. Here i am giving<br class="">
you only one error message from a node.<br class="">
sacct: error: NodeNames=radonc01 CPUs=32 doesn't match<br class="">
Sockets*CoresPerSocket*ThreadsPerCore (16), resetting CPUs<br class="">
The interesting thing is if you follow the<br class="">
Sockets*CoresPerSocket*ThreadsPerCore formula 2x8x2 = 32 however look above<br class="">
and it says (16) - Strange, no ?<br class="">
</blockquote>
<br class="">
No, Slurm is right. CPUS != threads. You've got 16 CPU cores, each with 2 <br class="">
threads. So in this configuration you can schedule 16 tasks per node and each <br class="">
task can use 2 threads.<br class="">
<br class="">
What does "slurmd -C" say on that node?<br class="">
<br class="">
All the best,<br class="">
Chris<br class="">
-- <br class="">
Chris Samuel : <a href="http://www.csamuel.org/" class="">http://www.csamuel.org/</a> : Melbourne, VIC<br class="">
<br class="">
<br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>