[slurm-users] cgroup2

[Christoph Brüning]

Hi everyone,

sorry to bring this old thread back up, but I'm seeing the same on 
Debian Bullseye with Slurm 20.11.3 from Debian's Repository.

Also thanks, Richard, for posting your solution. You're actually "not 
the only one stumbling on this." :)

I haven't found anything on that topic in the Release Notes of Slurm 
20.02 and 20.11, so I guess nothing of importance has changed from 19.05.

Now, what is the best practice in this case?
Switch back to hybrid cgroups hierarchy and wait for a future Slurm 
version that can handle unified?
Does anyone know if that version is being worked upon and maybe when we 
can expect it?

Cheers,
Christoph




On 07/06/2020 12.23, Richard Neuböck wrote:
> To answer my own question just in case I'm not the only one stumbling on
> this.
> 
> slurm 19.05 (packaged with Fedora 32) can't handle cgroup v2.
> 
> If there is nothing depending on cgroup v2 you can switch to the legacy
> hierarchy by adding a kernel boot command line:
> 
> systemd.unified_cgroup_hierarchy=0
> 
> 
> On 29.05.20 13:15, Richard Neuböck wrote:
>> Hi,
>>
>> I have a (hopefully quick) question: Do the slurm 19.05 cgroup plugins
>> know how to handle cgroup v2?
>>
>> I'm setting up slurm on Fedora 32 and get the following error messages:
>>
>> May 29 11:34:17 regulus slurmd[171632]: error: unable to mount cpuset
>> cgroup namespace: Device or resource busy
>> May 29 11:34:17 regulus slurmd[171632]: error: task/cgroup: unable to
>> create cpuset namespace
>> May 29 11:34:17 regulus slurmd[171632]: error: Couldn't load specified
>> plugin name for task/cgroup: Plugin init() callback failed
>> May 29 11:34:17 regulus slurmd[171632]: error: cannot create task
>> context for task/cgroup
>> May 29 11:34:17 regulus slurmd[171632]: error: slurmd initialization failed
>>
>> Google didn't reveal anything useful for me so my searching tangents and
>> parts of the slurm source just gave me some directions. I'm guessing
>> slurm only knows cgroup v1 so it fails when it tries to interact with
>> cgorup v2. Am I correct or am I barking up the wrong tree?
>>
>> Thanks for you feedback in advance!
>> Cheers
>> Richard
>>
>>
> 

-- 
Dr. Christoph Brüning
Universität Würzburg
HPC & DataManagement @ ct.qmat & RZUW
Am Hubland
D-97074 Würzburg
Tel.: +49 931 31-80499