<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Wes,</p>
<p>You didn't list the Slurm command that you used to get your interactive session. In particular did you ask Slurm for access to all 14 cores?
<br>
</p>
<p>Also note that since Matlab is using threads to distribute work among cores you don't want to ask for multiple tasks (-n or --ntasks) as that will give every process a single core. You want to allocate multiple CPUs per process with -c or --cpus-per-task
to tell Slurm how many CPUs your Matlab process should have access to.</p>
<p>Mike<br>
</p>
<div class="moz-cite-prefix">On 12/6/18 10:05 AM, Anderson, Wes R wrote:<br>
</div>
<blockquote type="cite" cite="mid:E1gUx6X-0006k1-VP@lists.schedmd.com">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p><strong>PERSONAL/NONWORK // EXTERNAL</strong><o:p></o:p></p>
<p class="MsoNormal">I took a look through the archives, and I did not see an clear answer to the issue I was seeing, so I thought I would go ahead and ask.<br>
<br>
I am having a cluster issue with SLURM and I hoped you might be able to help me out. I built a small test cluster to determine if it might meet some compute needs I have but seem to keep running into an issue where SLURM is restricting MATLAB to using a single
CPU regardless of how many we request. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">During testing I found the following:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">When I login into a MATLAB interactive session and run “feature numcores”<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I get the following:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><img style="width:3.9791in;height:2.3854in" id="_x0000_i1025" src="cid:part1.58B9958D.E8EDCF25@mines.edu" alt="cid:image002.jpg@01D48D35.180C4F30" class="" width="382" height="229"><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Which is correct, as I have 14 cores and they are all available.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">However when I go into SLURM and request a MATLAB interactive session and run the same command on the same computer:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><img style="width:9.802in;height:3.7291in" id="_x0000_i1026" src="cid:part2.754947F5.64E9532B@mines.edu" alt="cid:image004.png@01D48D35.180C4F30" class="" width="941" height="358"><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">So, what I understand is that my cgroups settings in SLURM are restricting MATLAB to a single core. Is that correct? Also, how do I fix this?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Here is my cgroups.conf<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">###<o:p></o:p></p>
<p class="MsoNormal">#<o:p></o:p></p>
<p class="MsoNormal"># Slurm cgroup support configuration file<o:p></o:p></p>
<p class="MsoNormal">#<o:p></o:p></p>
<p class="MsoNormal"># See man slurm.conf and man cgroup.conf for further<o:p></o:p></p>
<p class="MsoNormal"># information on cgroup configuration parameters<o:p></o:p></p>
<p class="MsoNormal">#--<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">########################################################<o:p></o:p></p>
<p class="MsoNormal"># W A R N I N G: This file is managed by Puppet #<o:p></o:p></p>
<p class="MsoNormal"># - - - - - - - changes are likely to be overwritten #<o:p></o:p></p>
<p class="MsoNormal">########################################################<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">#######################<o:p></o:p></p>
<p class="MsoNormal">CgroupAutomount=yes<o:p></o:p></p>
<p class="MsoNormal">#######################<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># testing -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-<o:p></o:p></p>
<p class="MsoNormal">#ConstrainCores=no<o:p></o:p></p>
<p class="MsoNormal">#ConstrainRAMSpace=no<o:p></o:p></p>
<p class="MsoNormal">#ConstrainSwapSpace=no<o:p></o:p></p>
<p class="MsoNormal"># testing -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">ConstrainCores=yes<o:p></o:p></p>
<p class="MsoNormal">ConstrainRAMSpace=yes<o:p></o:p></p>
<p class="MsoNormal">ConstrainSwapSpace=yes<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">#<o:p></o:p></p>
<p class="MsoNormal">ConstrainDevices=no<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">AllowedSwapSpace=0<o:p></o:p></p>
<p class="MsoNormal">MaxRAMPercent=100<o:p></o:p></p>
<p class="MsoNormal">MaxSwapPercent=100<o:p></o:p></p>
<p class="MsoNormal">##MinRAMSpace=30<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># TASK/CGROUP PLUGIN<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># Constrain the job cgroup RAM to this percentage of the allocated memory.<o:p></o:p></p>
<p class="MsoNormal">#AllowedRAMSpace=10<o:p></o:p></p>
<p class="MsoNormal">AllowedRamSpace=100<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># TaskAffinity=<yes|no><o:p></o:p></p>
<p class="MsoNormal"># If configured to "yes" then set a default task affinity to bind each<o:p></o:p></p>
<p class="MsoNormal"># step task to a subset of the allocated cores using<o:p></o:p></p>
<p class="MsoNormal"># sched_setaffinity. The default value is "no". Note: This feature<o:p></o:p></p>
<p class="MsoNormal"># requires the Portable Hardware Locality (hwloc) library to be<o:p></o:p></p>
<p class="MsoNormal"># installed.<o:p></o:p></p>
<p class="MsoNormal">TaskAffinity=yes<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"># MemorySwappiness=<number><o:p></o:p></p>
<p class="MsoNormal"># Configure the kernel's priority for swapping out anonymous pages (such as program data)<o:p></o:p></p>
<p class="MsoNormal"># verses file cache pages for the job cgroup. Valid values are between 0 and 100, inclusive. A<o:p></o:p></p>
<p class="MsoNormal"># value of 0 prevents the kernel from swapping out program data. A value of 100 gives equal<o:p></o:p></p>
<p class="MsoNormal"># priorioty to swapping out file cache or anonymous pages. If not set, then the kernel's default<o:p></o:p></p>
<p class="MsoNormal"># swappiness value will be used. Either ConstrainRAMSpace or ConstrainSwapSpace must<o:p></o:p></p>
<p class="MsoNormal"># be set to yes in order for this parameter to be applied.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">MemorySwappiness=0<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">#####################################################################################<o:p></o:p></p>
<p class="MsoNormal"># If compute nodes mount Lustre or NFS file systems, it may be a good idea to #<o:p></o:p></p>
<p class="MsoNormal"># configure cgroup.conf with: #<o:p></o:p></p>
<p class="MsoNormal"># ConstrainKmemSpace=no #<o:p></o:p></p>
<p class="MsoNormal"># #<o:p></o:p></p>
<p class="MsoNormal"># From <a class="moz-txt-link-rfc2396E" href="https://wiki.fysik.dtu.dk/niflheim/Slurm_configuration#activating-cgroups">
<https://wiki.fysik.dtu.dk/niflheim/Slurm_configuration#activating-cgroups></a> #<o:p></o:p></p>
<p class="MsoNormal">#####################################################################################<o:p></o:p></p>
<p class="MsoNormal">ConstrainKmemSpace=no #<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">########################################################<o:p></o:p></p>
<p class="MsoNormal"># W A R N I N G: This file is managed by Puppet #<o:p></o:p></p>
<p class="MsoNormal"># - - - - - - - changes are likely to be overwritten #<o:p></o:p></p>
<p class="MsoNormal">########################################################<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
<p class="MsoNormal">Wes<o:p></o:p></p>
<p class="MsoNormal">(A slurm neophyte)<o:p></o:p></p>
</div>
</blockquote>
</body>
</html>