Hi Jon,
Did you check the software prerequisite minimum versions listed in this page: https://slurm.schedmd.com/rest_quickstart.html#prereq
IHTH, Ole
On 8/13/2025 4:28 PM, Jon Marshall via slurm-users wrote:
Hi,
I feel like I must have missed something obvious here, so hopefully someone can spot it! We recently updated to Slurm 24.11.6 on our cluster and I made sure to compile --with slurmrestd and --with jwt, as there are a few things I've been meaning to try out with the REST API.
From the rpmbuild config.log I get:
$ ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat- linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/ etc/slurm --datadir=/usr/share --includedir=/usr/include --libdir=/usr/ lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/ var/lib --mandir=/usr/share/man --infodir=/usr/share/info --with- systemdsystemunitdir=/usr/lib/systemd/system --enable-pkgconfig --with-jwt -- configure:21020: checking for jwt.h configure:21020: gcc -c -DNUMA_VERSION1_COMPATIBILITY -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,- D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc- switches -Wl,-z,lazy -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection - fcf-protection -fno-omit-frame-pointer conftest.c >&5 configure:21020: $? = 0 configure:21020: result: yes configure:21028: checking for jwt_add_header in -ljwt configure:21051: gcc -o conftest -DNUMA_VERSION1_COMPATIBILITY -O2 -g - pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,- D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc- switches -Wl,-z,lazy -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection - fcf-protection -fno-omit-frame-pointer -Wl,-z,relro -Wl,-z,lazy conftest.c -ljwt -ldl -lrt -lm -lresolv >&5 configure:21051: $? = 0 configure:21061: result: yes -- config.status:1746: creating src/plugins/auth/jwt/Makefile -- config.status:1746: creating src/slurmrestd/plugins/auth/jwt/Makefile -- ac_cv_header_jwt_h=yes -- ac_cv_lib_jwt_jwt_add_header=yes -- JWT_LDFLAGS='-ljwt'
And:
configure:26873: checking whether to compile slurmrestd configure:26904: result: yes -- configure:26915: checking for slurmrestd default port configure:26933: result: 6820
And indeed I get a slurmrestd package out of it, which I've successfully installed with no errors. I've created a JWT key and rolled it out to the controller node, but when I go to check auth types I get:
slurmrestd -a rest_auth/jwt slurmrestd: fatal: Unable to find plugin: rest_auth/jwt
Running an strace on this process shows that it does in fact stat the .so that does in fact exist:
stat("/usr/lib64/slurm/auth_jwt.so", {st_mode=S_IFREG|0755, st_size=283728, ...}) = 0
And I'm kind of stumped now. I've checked through all the resolved bug tickets to see if I can spot what is wrong but I'm getting nothing, and I can't find anything in the mailing lists either, other than things I've checked. Has anyone else had similar issues, or can someone see the blindingly obvious thing I'm missing?