<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hello,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
We decided to route all jobs requesting from 1 to 20 cores to our serial queue. Furthermore, the nodes controlled by the serial queue are shared by multiple users. We did this to try to reduce the level of fragmentation across the cluster -- our default "batch"
queue provides exclusive access to compute nodes. </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
It looks like the downside of the serial queue is that jobs from different users can interact quite badly. To some extent this is an education issue -- for example matlab users need to be told to add the "-singleCompThread" option to their command line. On
the other hand I wonder if our cgroups setup is optimal for the serial queue. Our
<b>cgroup.conf</b> contains...</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span><b>CgroupAutomount=yes<br>
</b></span>
<div><b>CgroupReleaseAgentDir="/etc/slurm/cgroup"<br>
</b></div>
<div><b><br>
</b></div>
<div><b>ConstrainCores=yes<br>
</b></div>
<div><b>ConstrainRAMSpace=yes<br>
</b></div>
<div><b>ConstrainDevices=yes<br>
</b></div>
<div><b>TaskAffinity=no<br>
</b></div>
<div><b><br>
</b></div>
<span><b>CgroupMountpoint=/sys/fs/cgroup</b></span><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span><br>
</span></div>
<div style=""><font color="#000000" face="Calibri, Arial, Helvetica, sans-serif"><span style="font-size: 12pt;">The
</span>relevant<span style="font-size: 12pt;"> cgroup configuration in the <b>slurm.conf
</b>is...</span></font></div>
<div style=""><b><span>ProctrackType=proctrack/cgroup<br>
</span><span>TaskPlugin=affinity,cgroup</span></b><br>
</div>
<div style=""><br>
</div>
<div style="">Could someone please advise us on the required/recommended cgroup setup for the above scenario? For example, should we really set "TaskAffinity=yes"? I assume the interaction between jobs (sometimes jobs can get stalled) is due to context switching
at the kernel level, however (apart from educating users) how can we minimise that switching on the serial nodes?</div>
<div style=""><br>
</div>
<div style="">Best regards,</div>
<div style="">David</div>
<div style=""><br>
</div>
</body>
</html>