Hello,
I have encountered issues with running slurmctld.
From logs, I see these errors:
[2024-06-14T17:37:57.587] slurmctld version 24.05.0 started on cluster laura [2024-06-14T17:37:57.587] error: plugin_load_from_file: dlopen(/usr/lib64/slurm/jobacct_gather_cgroup.so): /usr/lib64/slurm/jobacct_gather_cgroup.so: undefined symbol: xcpuinfo_init [2024-06-14T17:37:57.587] error: Couldn't load specified plugin name for jobacct_gather/cgroup: Dlopen of plugin file failed [2024-06-14T17:37:57.587] error: cannot create jobacct_gather context for jobacct_gather/cgroup [2024-06-14T17:37:57.587] fatal: failed to initialize jobacct_gather plugin [2024-06-14T17:39:07.741] Not running as root. Can't drop supplementary groups
Aftre setting
#JobAcctGatherType=
the problem changed to:
[2024-06-14T17:39:07.742] slurmctld version 24.05.0 started on cluster laura [2024-06-14T17:39:07.742] error: plugin_load_from_file: dlopen(/usr/lib64/slurm/prep_script.so): /usr/lib64/slurm/prep_script.so: undefined symbol: send_slurmd_conf_lite [2024-06-14T17:39:07.742] error: Couldn't load specified plugin name for prep/script: Dlopen of plugin file failed [2024-06-14T17:39:07.742] error: prep_g_init: cannot create prep context for prep/script [2024-06-14T17:39:07.742] fatal: failed to initialize prep plugin
I also tried that with git-master:
[2024-06-14T17:48:21.691] Not running as root. Can't drop supplementary groups [2024-06-14T17:48:21.691] error: Job accounting information gathered, but not stored [2024-06-14T17:48:21.692] slurmctld version 24.11.0-0rc1 started on cluster laura [2024-06-14T17:48:21.692] error: plugin_load_from_file: dlopen(/usr/lib64/slurm/jobacct_gather_cgroup.so): /usr/lib64/slurm/jobacct_gather_cgroup.so: undefined symbol: xcpuinfo_init [2024-06-14T17:48:21.692] error: Couldn't load specified plugin name for jobacct_gather/cgroup: Dlopen of plugin file failed [2024-06-14T17:48:21.692] error: cannot create jobacct_gather context for jobacct_gather/cgroup [2024-06-14T17:48:21.692] fatal: failed to initialize jobacct_gather plugin
Any idea what may be wrong?
Regards, Rafał
On 14.06.2024 17:51, Rafał Lalik via slurm-users wrote:
Hello,
I have encountered issues with running slurmctld.
From logs, I see these errors:
[2024-06-14T17:37:57.587] slurmctld version 24.05.0 started on cluster laura [2024-06-14T17:37:57.587] error: plugin_load_from_file: dlopen(/usr/lib64/slurm/jobacct_gather_cgroup.so): /usr/lib64/slurm/jobacct_gather_cgroup.so: undefined symbol: xcpuinfo_init [2024-06-14T17:37:57.587] error: Couldn't load specified plugin name for jobacct_gather/cgroup: Dlopen of plugin file failed [2024-06-14T17:37:57.587] error: cannot create jobacct_gather context for jobacct_gather/cgroup [2024-06-14T17:37:57.587] fatal: failed to initialize jobacct_gather plugin [2024-06-14T17:39:07.741] Not running as root. Can't drop supplementary groups
Aftre setting
#JobAcctGatherType=
the problem changed to:
[2024-06-14T17:39:07.742] slurmctld version 24.05.0 started on cluster laura [2024-06-14T17:39:07.742] error: plugin_load_from_file: dlopen(/usr/lib64/slurm/prep_script.so): /usr/lib64/slurm/prep_script.so: undefined symbol: send_slurmd_conf_lite [2024-06-14T17:39:07.742] error: Couldn't load specified plugin name for prep/script: Dlopen of plugin file failed [2024-06-14T17:39:07.742] error: prep_g_init: cannot create prep context for prep/script [2024-06-14T17:39:07.742] fatal: failed to initialize prep plugin
I also tried that with git-master:
[2024-06-14T17:48:21.691] Not running as root. Can't drop supplementary groups [2024-06-14T17:48:21.691] error: Job accounting information gathered, but not stored [2024-06-14T17:48:21.692] slurmctld version 24.11.0-0rc1 started on cluster laura [2024-06-14T17:48:21.692] error: plugin_load_from_file: dlopen(/usr/lib64/slurm/jobacct_gather_cgroup.so): /usr/lib64/slurm/jobacct_gather_cgroup.so: undefined symbol: xcpuinfo_init [2024-06-14T17:48:21.692] error: Couldn't load specified plugin name for jobacct_gather/cgroup: Dlopen of plugin file failed [2024-06-14T17:48:21.692] error: cannot create jobacct_gather context for jobacct_gather/cgroup [2024-06-14T17:48:21.692] fatal: failed to initialize jobacct_gather plugin
Any idea what may be wrong?
Recent compiler-hardening efforts broke slurms way of loading plugins. As a workaround, link slurm with -Wl,-z,lazy
Recent compiler-hardening efforts broke slurms way of loading plugins. As a workaround, link slurm with -Wl,-z,lazy
-- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-leave@lists.schedmd.com
Thanks, this fixed issue for me.
Regards, Rafał