<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Double-check you have all the packages.</p>
<p>When slurm is built, slurmrestd is a separate package and is only
built if the whole set was directed to do so. If they did not
build it, you will need to do so yourself. This will mean using
your custom built files throughout.</p>
<p>Brian Andrus<br>
</p>
<div class="moz-cite-prefix">On 3/7/2022 3:04 PM, Justin Freeman
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:53749576-AB85-4C76-B164-25FCAE428A8B@bom.gov.au">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style>
<div class="WordSection1">
<p class="MsoNormal">Hi slurm-users,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hoping you can point me in the right
direction….I am trying to setup the Slurm REST API under
parallelcluster v3.1.1. I am following the instructions at
<a class="moz-txt-link-freetext" href="https://slurm.schedmd.com/jwt.html">https://slurm.schedmd.com/jwt.html</a>. I have added the
AuthAltTypes=rest_auth/jwt and the
AuthAultParameters=jwt_key=… to /opt/slum/etc/slurm.conf but
restarting slurmctld fails with:
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">slurmctld: debug: slurmctld log levels:
stderr=debug3 logfile=debug3 syslog=quiet<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug: Log file re-opened<o:p></o:p></p>
<p class="MsoNormal">slurmctld: pidfile not locked, assuming no
running daemon<o:p></o:p></p>
<p class="MsoNormal">slurmctld: Stack size set to 10485760<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug: slurmscriptd: Got ack
from slurmctld, initialization successful<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug: slurmctld: slurmscriptd
fork()'d and initialized.<o:p></o:p></p>
<p class="MsoNormal">slurmctld: slurmctld version 21.08.5
started on cluster parallelcluster<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Trying to load plugin
/opt/slurm/lib/slurm/cred_munge.so<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug: _slurmscriptd_mainloop:
started<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Called _msg_readable<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug:
_slurmctld_listener_thread: started listening to slurmscriptd<o:p></o:p></p>
<p class="MsoNormal">slurmctld: cred/munge: init: Munge
credential signature plugin loaded<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Success.<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Called _msg_readable<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Trying to load plugin
/opt/slurm/lib/slurm/auth_munge.so<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug: auth/munge: init: Munge
authentication plugin loaded<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Success.<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Trying to load plugin
/opt/slurm/lib/slurm/rest_auth_jwt.so<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Couldn't find sym
'auth_p_create' in the plugin<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Couldn't find sym
'auth_p_destroy' in the plugin<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Couldn't find sym
'auth_p_verify' in the plugin<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Couldn't find sym
'auth_p_get_uid' in the plugin<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Couldn't find sym
'auth_p_get_gid' in the plugin<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Couldn't find sym
'auth_p_get_host' in the plugin<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Couldn't find sym
'auth_p_pack' in the plugin<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Couldn't find sym
'auth_p_unpack' in the plugin<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Couldn't find sym
'auth_p_thread_config' in the plugin<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Couldn't find sym
'auth_p_thread_clear' in the plugin<o:p></o:p></p>
<p class="MsoNormal">slurmctld: debug3: Couldn't find sym
'auth_p_token_generate' in the plugin<o:p></o:p></p>
<p class="MsoNormal">slurmctld: fatal: plugin_load_and_link:
Plugin loading failed due to missing symbols. Plugin is
corrupted.<br>
<br>
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Looks like I am missing something when
trying to enable JWT authentication with the slurm services. I
am using the slurm install that comes out of the box with
parallelcluster 3.1.1. I was wondering if there are any
suggestions about how to fix/resolve this error?. Thanks. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Slurmrestd auth types:<o:p></o:p></p>
<p class="MsoNormal">$ slurmrestd -a list<o:p></o:p></p>
<p class="MsoNormal">slurmrestd: Possible REST authentication
plugins:<o:p></o:p></p>
<p class="MsoNormal">slurmrestd: rest_auth/local<o:p></o:p></p>
<p class="MsoNormal">slurmrestd: rest_auth/jwt<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Also, step 2 in the JWT instructions at <a
href="https://slurm.schedmd.com/jwt.html"
moz-do-not-send="true" class="moz-txt-link-freetext">
https://slurm.schedmd.com/jwt.html</a> is a little confusing
– the key is saved to the StateSaveLocation but the rest of
the instructions reference /etc/slurm? Am I missing something
here (likely!) ?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">dd if=/dev/random
of=/var/spool/slurm/statesave/jwt_hs256.key bs=32 count=1<o:p></o:p></p>
<p class="MsoNormal">chown slurm:slurm /etc/slurm/jwt_hs256.key<o:p></o:p></p>
<p class="MsoNormal">chmod 0600 /etc/slurm/jwt_hs256.key<o:p></o:p></p>
<p class="MsoNormal">chown root:root /etc/slurm<o:p></o:p></p>
<p class="MsoNormal">chmod 0755 /etc/slurm<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks!<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Regards,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">j<o:p></o:p></p>
</div>
</blockquote>
</body>
</html>