Hi,
I feel like I must have missed something obvious here, so hopefully someone can spot it! We recently updated to Slurm 24.11.6 on our cluster and I made sure to compile --with slurmrestd and --with jwt, as there are a few things I've been meaning to try out with the REST API.
From the rpmbuild config.log I get:
$ ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc/slurm --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-systemdsystemunitdir=/usr/lib/systemd/system --enable-pkgconfig --with-jwt -- configure:21020: checking for jwt.h configure:21020: gcc -c -DNUMA_VERSION1_COMPATIBILITY -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -Wl,-z,lazy -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer conftest.c >&5 configure:21020: $? = 0 configure:21020: result: yes configure:21028: checking for jwt_add_header in -ljwt configure:21051: gcc -o conftest -DNUMA_VERSION1_COMPATIBILITY -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -Wl,-z,lazy -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -Wl,-z,relro -Wl,-z,lazy conftest.c -ljwt -ldl -lrt -lm -lresolv >&5 configure:21051: $? = 0 configure:21061: result: yes -- config.status:1746: creating src/plugins/auth/jwt/Makefile -- config.status:1746: creating src/slurmrestd/plugins/auth/jwt/Makefile -- ac_cv_header_jwt_h=yes -- ac_cv_lib_jwt_jwt_add_header=yes -- JWT_LDFLAGS='-ljwt'
And:
configure:26873: checking whether to compile slurmrestd configure:26904: result: yes -- configure:26915: checking for slurmrestd default port configure:26933: result: 6820
And indeed I get a slurmrestd package out of it, which I've successfully installed with no errors. I've created a JWT key and rolled it out to the controller node, but when I go to check auth types I get:
slurmrestd -a rest_auth/jwt slurmrestd: fatal: Unable to find plugin: rest_auth/jwt
Running an strace on this process shows that it does in fact stat the .so that does in fact exist:
stat("/usr/lib64/slurm/auth_jwt.so", {st_mode=S_IFREG|0755, st_size=283728, ...}) = 0
And I'm kind of stumped now. I've checked through all the resolved bug tickets to see if I can spot what is wrong but I'm getting nothing, and I can't find anything in the mailing lists either, other than things I've checked. Has anyone else had similar issues, or can someone see the blindingly obvious thing I'm missing?
Cheers Jon
Jon Marshall
High Performance Computing Specialist
IT and Scientific Computing Team
Cancer Research UK Cambridge Institute
Li Ka Shing Centre | Robinson Way | Cambridge | CB2 0RE
Webhttp://www.cruk.cam.ac.uk/ | Facebookhttp://www.facebook.com/cancerresearchuk | Twitterhttp://twitter.com/CR_UK
[Description: CRI Logo]http://www.cruk.cam.ac.uk/
Hi Jon,
Did you check the software prerequisite minimum versions listed in this page: https://slurm.schedmd.com/rest_quickstart.html#prereq
IHTH, Ole
On 8/13/2025 4:28 PM, Jon Marshall via slurm-users wrote:
Hi,
I feel like I must have missed something obvious here, so hopefully someone can spot it! We recently updated to Slurm 24.11.6 on our cluster and I made sure to compile --with slurmrestd and --with jwt, as there are a few things I've been meaning to try out with the REST API.
From the rpmbuild config.log I get:
$ ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat- linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/ etc/slurm --datadir=/usr/share --includedir=/usr/include --libdir=/usr/ lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/ var/lib --mandir=/usr/share/man --infodir=/usr/share/info --with- systemdsystemunitdir=/usr/lib/systemd/system --enable-pkgconfig --with-jwt -- configure:21020: checking for jwt.h configure:21020: gcc -c -DNUMA_VERSION1_COMPATIBILITY -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,- D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc- switches -Wl,-z,lazy -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection - fcf-protection -fno-omit-frame-pointer conftest.c >&5 configure:21020: $? = 0 configure:21020: result: yes configure:21028: checking for jwt_add_header in -ljwt configure:21051: gcc -o conftest -DNUMA_VERSION1_COMPATIBILITY -O2 -g - pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,- D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc- switches -Wl,-z,lazy -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection - fcf-protection -fno-omit-frame-pointer -Wl,-z,relro -Wl,-z,lazy conftest.c -ljwt -ldl -lrt -lm -lresolv >&5 configure:21051: $? = 0 configure:21061: result: yes -- config.status:1746: creating src/plugins/auth/jwt/Makefile -- config.status:1746: creating src/slurmrestd/plugins/auth/jwt/Makefile -- ac_cv_header_jwt_h=yes -- ac_cv_lib_jwt_jwt_add_header=yes -- JWT_LDFLAGS='-ljwt'
And:
configure:26873: checking whether to compile slurmrestd configure:26904: result: yes -- configure:26915: checking for slurmrestd default port configure:26933: result: 6820
And indeed I get a slurmrestd package out of it, which I've successfully installed with no errors. I've created a JWT key and rolled it out to the controller node, but when I go to check auth types I get:
slurmrestd -a rest_auth/jwt slurmrestd: fatal: Unable to find plugin: rest_auth/jwt
Running an strace on this process shows that it does in fact stat the .so that does in fact exist:
stat("/usr/lib64/slurm/auth_jwt.so", {st_mode=S_IFREG|0755, st_size=283728, ...}) = 0
And I'm kind of stumped now. I've checked through all the resolved bug tickets to see if I can spot what is wrong but I'm getting nothing, and I can't find anything in the mailing lists either, other than things I've checked. Has anyone else had similar issues, or can someone see the blindingly obvious thing I'm missing?
-
Jon Marshall -
Ole Holm Nielsen