<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto">Indeed, the issue seems to be that Ubuntu 22.04 does not support cgroups v1 anymore. Does SLURM support cgroupsv2? It seems so: <a href="https://slurm.schedmd.com/cgroup_v2.html">https://slurm.schedmd.com/cgroup_v2.html</a><br><br><div dir="ltr">/Abel</div><div dir="ltr"><br><blockquote type="cite">On Jun 15, 2023, at 20:20, Reed Dier <reed.dier@focusvq.com> wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr"><meta http-equiv="Content-Type" content="text/html; charset=utf-8">I don’t have any direct advice off-hand, but I figure I will try to help steer the conversation in the right direction for figuring it out.<div class=""><br class=""></div><div class="">I’m going to assume that since you mention 21.08.5, that this means you are using the slurm-wlm packages from the ubuntu repos, and not building yourself?</div><div class=""><br class=""></div><div class="">And have all the components (slurmctld(s), slurmdbd, slurmd(s)) been upgraded as well?</div><div class=""><br class=""></div><div class=""><div>The only thing that immediately comes to mind is that I remember reading a good bit about Ubuntu 22.04’s use of cgroups v2, which as I understand it are very different from cgroups v1, and plenty of people have had issues with v1/v2 mismatches with slurm and other applications.</div><div><br class=""></div><div><a href="https://www.reddit.com/r/SLURM/comments/vjquih/error_cannot_find_cgroup_plugin_for_cgroupv2/" class="">https://www.reddit.com/r/SLURM/comments/vjquih/error_cannot_find_cgroup_plugin_for_cgroupv2/</a></div><div><a href="https://groups.google.com/g/slurm-users/c/0dJhe5r6_2Q?pli=1" class="">https://groups.google.com/g/slurm-users/c/0dJhe5r6_2Q?pli=1</a></div><div><a href="https://discuss.linuxcontainers.org/t/after-updated-to-more-recent-ubuntu-version-with-cgroups-v2-ubuntu-16-04-container-is-not-working-properly/14022" class="">https://discuss.linuxcontainers.org/t/after-updated-to-more-recent-ubuntu-version-with-cgroups-v2-ubuntu-16-04-container-is-not-working-properly/14022</a></div><div><br class=""></div><div>Hope that at least steers the conversation in a good direction.</div><div><br class=""></div><div>Reed</div><div><br class=""><blockquote type="cite" class=""><div class="">On Jun 15, 2023, at 5:04 PM, Tim Schneider <<a href="mailto:tim.schneider1@tu-darmstadt.de" class="">tim.schneider1@tu-darmstadt.de</a>> wrote:</div><br class="Apple-interchange-newline"><div class="">
<meta http-equiv="content-type" content="text/html; charset=UTF-8" class="">
<div class="">
Hi,
<div class="moz-forward-container"><p class="">I am maintaining the SLURM cluster of my research group.
Recently I updated to Ubuntu 22.04 and Slurm 21.08.5 and ever
since, I am unable to launch jobs. When launching a job, I
receive the following error:</p><p class=""><i class="">$ srun --nodes=1 --ntasks-per-node=1 -c 1 --mem-per-cpu 1G
--time=01:00:00 --pty -p amd -w cn02 --pty bash -i</i><i class=""><br class="">
</i><i class="">srun: error: task 0 launch failed: Plugin initialization
failed</i></p><p class="">Strangely, I cannot find any indication of this problem in the
logs (find the logs attached). The problem must be related to
the task/cgroup plugin, as it does not occur when I disable it.</p><p class="">After reading in the documentation, I tried adding the <i class="">cgroup_enable=memory
swapaccount=1</i> kernel parameters, but the problem
persisted.</p><p class="">I would be very grateful for any advice where to look since I
have no idea how to investigate this issue further.</p><p class="">Thanks a lot in advance.</p><p class="">Best,</p><p class="">Tim<br class="">
</p><p class=""><br class="">
</p>
</div>
</div>
<span id="cid:20813A19-49FA-477E-A30F-A1870FA53F1A"><cgroup.conf></span><span id="cid:6EA7789A-0BBC-46F4-A871-21FF5344DAA2"><slurmd.log></span><span id="cid:1BEA19F6-B8FD-4E05-96A4-E00AB43D54F6"><slurmctld.log></span></div></blockquote></div><br class=""></div></div></blockquote></body></html>