[slurm-users] can't create memory group (cgroup)

[Mike Cammilleri]

Hi everyone,

I'm getting this error lately for everyone's jobs, which results in memory not being constrained via the cgroups plugin.


slurmstepd: error: task/cgroup: unable to add task[pid=21681] to memory cg '(null)'
slurmstepd: error: jobacct_gather/cgroup: unable to instanciate user 3691 memory cgroup

The result is that no uid_ direcotries are created under /sys/fs/cgroup/memory


Here is our cgroup.conf file:

CgroupAutomount=yes
CgroupReleaseAgentDir="/etc/cgroup"
CgroupMountpoint=/sys/fs/cgroup
ConstrainCores=yes
ConstrainDevices=no
ConstrainRAMSpace=yes
ConstrainSwapSpace=yes
AllowedSwapSpace=0

We are using jobacct_gather/cgroup
# ACCOUNTING
JobAcctGatherType=jobacct_gather/cgroup

The partition is configured like this
PartitionName=long Nodes=marzano[05-13] PriorityTier=30 Default=NO MaxTime=5-0 State=UP OverSubscribe=FORCE:1

We are using slurm 16.05.6 on Ubuntu 14.04 LTS

Any ideas how to get cgroups going again?