<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Hi David,<br>
<br>
doing it the way you do it, is the same way, we do it.<br>
<br>
When the Matlab job asks for one CPU, it only gets on CPU this way.
That means, that all the processes are bound to this one CPU. So
(theoretically) the user is just disturbing himself, if he uses
more.<br>
<br>
But especially Matlab, there are more things to do. I t does not
suffice to add '-singleCompThread' to the commandline. Matlab is not
the only tool, that tries to use all cores, it finds on the node.<br>
The same is valid for CPLEX and Gurobi, both often used from Matlab.
So even, if the user sets '-singleCompThread' for Matlab, that does
not mean at all, the job is only using one CPU.<br>
<br>
<br>
Best<br>
Marcus<br>
<br>
<div class="moz-cite-prefix">On 11/4/19 4:14 PM, David Baker wrote:<br>
</div>
<blockquote type="cite"
cite="mid:LO2P265MB0381FAF73B3E3CC40A54C3A2FE7F0@LO2P265MB0381.GBRP265.PROD.OUTLOOK.COM">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
Hello,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
We decided to route all jobs requesting from 1 to 20 cores to
our serial queue. Furthermore, the nodes controlled by the
serial queue are shared by multiple users. We did this to try to
reduce the level of fragmentation across the cluster -- our
default "batch" queue provides exclusive access to compute
nodes. </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
It looks like the downside of the serial queue is that jobs from
different users can interact quite badly. To some extent this is
an education issue -- for example matlab users need to be told
to add the "-singleCompThread" option to their command line. On
the other hand I wonder if our cgroups setup is optimal for the
serial queue. Our
<b>cgroup.conf</b> contains...</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<span><b>CgroupAutomount=yes<br>
</b></span>
<div><b>CgroupReleaseAgentDir="/etc/slurm/cgroup"<br>
</b></div>
<div><b><br>
</b></div>
<div><b>ConstrainCores=yes<br>
</b></div>
<div><b>ConstrainRAMSpace=yes<br>
</b></div>
<div><b>ConstrainDevices=yes<br>
</b></div>
<div><b>TaskAffinity=no<br>
</b></div>
<div><b><br>
</b></div>
<span><b>CgroupMountpoint=/sys/fs/cgroup</b></span><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<span><br>
</span></div>
<div style=""><font face="Calibri, Arial, Helvetica, sans-serif"
color="#000000"><span style="font-size: 12pt;">The
</span>relevant<span style="font-size: 12pt;"> cgroup
configuration in the <b>slurm.conf
</b>is...</span></font></div>
<div style=""><b><span>ProctrackType=proctrack/cgroup<br>
</span><span>TaskPlugin=affinity,cgroup</span></b><br>
</div>
<div style=""><br>
</div>
<div style="">Could someone please advise us on the
required/recommended cgroup setup for the above scenario? For
example, should we really set "TaskAffinity=yes"? I assume the
interaction between jobs (sometimes jobs can get stalled) is due
to context switching at the kernel level, however (apart from
educating users) how can we minimise that switching on the
serial nodes?</div>
<div style=""><br>
</div>
<div style="">Best regards,</div>
<div style="">David</div>
<div style=""><br>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Marcus Wagner, Dipl.-Inf.
IT Center
Abteilung: Systeme und Betrieb
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Tel: +49 241 80-24383
Fax: +49 241 80-624383
<a class="moz-txt-link-abbreviated" href="mailto:wagner@itc.rwth-aachen.de">wagner@itc.rwth-aachen.de</a>
<a class="moz-txt-link-abbreviated" href="http://www.itc.rwth-aachen.de">www.itc.rwth-aachen.de</a>
</pre>
</body>
</html>