<div dir="ltr">Hi Matt,<div><br></div><div>As a protest to asking questions on this list and getting solicitations for pay-for support, let me give you some advice for free :)</div><div><br></div><div>If you look at your slurm.conf you'll see there are two directories that your slurm user and group need to have write access to.</div><div><br></div><div>One is whatever you configure as SlurmdSpoolDir. This needs to be available on all worker nodes that are running slurmd. Set ownership to slurm user and slurm group and mode 755.</div><div><br></div><div>The other is StateSaveLocation. This needs to be present just on your controller (where slurmctld runs). Again, this should have ownership of slurm user and slurm group and mode 755.</div><div><br></div><div>You probably want to use something more specific than just /var/spool for your StateSaveLocation</div><div><br>Best,</div><div><br></div><div>Sean</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Apr 11, 2018 at 1:48 PM, Jess Arrington <span dir="ltr"><<a href="mailto:jess@schedmd.com" target="_blank">jess@schedmd.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="color:rgb(11,83,148)">Hi Matt,</div><div class="gmail_default" style="color:rgb(11,83,148)"><br></div><div class="gmail_default"><font face="Arial" style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><font face="Arial"><font face="Arial"><font face="Arial" style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px">I hope your day is treating you well.<br><br></font><br><font face="Arial" style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px">Thank you for your posts on the Slurm user list. <span> </span></font><br><br><br><span style="font-size:12.8px">By chance, do you work with Paul Van Der Mark?</span></font></font></font></div><div class="gmail_default"><font face="Arial" style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><font face="Arial"><font face="Arial"><span style="font-size:12.8px"><br></span></font></font></font></div><div class="gmail_default"><font face="Arial" style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><font face="Arial"><font face="Arial"><span style="font-size:12.8px"><br>Would there be interest on your side to see a Slurm support contract for your systems at </span></font></font></font><font face="Arial"><span style="font-size:12.8px">FSU</span></font><span style="color:rgb(34,34,34);font-family:Arial;font-size:12.8px">?</span></div><div class="gmail_default" style="color:rgb(11,83,148)"><font face="Arial" style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><font face="Arial"><font face="Arial"><br></font></font></font><font face="Arial" style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><font face="Arial">Sites running Slurm with support give us feedback that support is invaluable and a great return back to the organization with much better system utilization with optimized configs by our experts<span> </span></font></font><font face="Arial" style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><font face="Arial"><span class="m_-4729733019118038510gmail-m_-6044234129629131185font" style="font-family:Arial"><font face="Arial"><font face="Arial">(which pays for the support contract in and of itself)<span> </span></font></font></span>and their sites not having to rely on in-house best effort support hacks that get very expensive and turn into complicated chaos and potential down systems.</font></font><font face="Arial" style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><font face="Arial"><br></font></font><font face="Arial" style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br><br><font face="Arial">Additionally, support keeps the Slurm project alive and going strong</font><span> </span> <span> </span><br><br><br>Please let me know your thoughts or if you would like me to reach out to another contact at FSU to chat about this further.</font><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"></span><div class="m_-4729733019118038510gmail-m_-6044234129629131185moz-signature" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br><br><br>Take care,</div><br></div><div class="gmail_default" style="color:rgb(11,83,148)"><br></div><div class="gmail_extra"><br clear="all"><div><div class="m_-4729733019118038510gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><b style="font-size:12.8px"><span></span><span></span>Jess Arrington</b><br></div><div dir="ltr">Director of Sales | 801-616-7823 <br><a href="https://maps.google.com/?q=204+N+1200+E+%23203+Lehi,+UT+84043&entry=gmail&source=g">204 N 1200 E #203 Lehi, UT 84043</a><br><br></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Wed, Apr 11, 2018 at 6:26 AM, Matt Hohmeister <span dir="ltr"><<a href="mailto:hohmeister@psy.fsu.edu" target="_blank">hohmeister@psy.fsu.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">





<div lang="EN-US">
<div class="m_-4729733019118038510gmail-m_8480027517044187288WordSection1">
<p class="MsoNormal">I’m brand-new to Slurm, and setting it up on a single RHEL 7.4 VM as a proof of concept before I deploy it. After following the instructions on
<a href="https://www.slothparadise.com/how-to-install-slurm-on-centos-7-cluster/" target="_blank">
https://www.slothparadise.com/<wbr>how-to-install-slurm-on-centos<wbr>-7-cluster/</a> (sorry, site not working now), I can get slurmd to start perfectly, but slurmctld fails to start with the following
<span style="font-family:Consolas">journalctl -xe</span>; I was wondering if anyone has run into this or could shed some light on this…thanks in advance!<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">Apr 11 08:18:30 psy-slurm polkitd[680]: Registered Authentication Agent for unix-process:1779:31362 (system bus name :1.26 [/usr/bin/pkttyagent --notify-fd 5 --fallbac<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">Apr 11 08:18:30 psy-slurm systemd[1]: Starting Slurm controller daemon...<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">-- Subject: Unit slurmctld.service has begun start-up<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">-- Defined-By: systemd<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">-- Support: <a href="http://lists.freedesktop.org/mailman/listinfo/systemd-devel" target="_blank">http://lists.freedesktop.org/m<wbr>ailman/listinfo/systemd-devel</a><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">--<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">-- Unit slurmctld.service has begun starting up.<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">Apr 11 08:18:30 psy-slurm systemd[1]: PID file /var/run/slurmctld.pid not readable (yet?) after start.<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">Apr 11 08:18:30 psy-slurm systemd[1]: Started Slurm controller daemon.<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">-- Subject: Unit slurmctld.service has finished start-up<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">-- Defined-By: systemd<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">-- Support: <a href="http://lists.freedesktop.org/mailman/listinfo/systemd-devel" target="_blank">http://lists.freedesktop.org/m<wbr>ailman/listinfo/systemd-devel</a><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">--<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">-- Unit slurmctld.service has finished starting up.<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">--<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">-- The start-up result is done.<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">Apr 11 08:18:30 psy-slurm polkitd[680]: Unregistered Authentication Agent for unix-process:1779:31362 (system bus name :1.26, object path /org/freedesktop/PolicyKit1/A<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">Apr 11 08:18:30 psy-slurm slurmctld[1787]: fatal: Incorrect permissions on state save loc: /var/spool<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">Apr 11 08:18:30 psy-slurm systemd[1]: slurmctld.service: main process exited, code=exited, status=1/FAILURE<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">Apr 11 08:18:30 psy-slurm systemd[1]: Unit slurmctld.service entered failed state.<u></u><u></u></span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-family:Consolas">Apr 11 08:18:30 psy-slurm systemd[1]: slurmctld.service failed.<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Matt Hohmeister<u></u><u></u></p>
<p class="MsoNormal">Systems and Network Administrator<u></u><u></u></p>
<p class="MsoNormal">Department of Psychology<u></u><u></u></p>
<p class="MsoNormal">Florida State University<u></u><u></u></p>
<p class="MsoNormal">PO Box 3064301<u></u><u></u></p>
<p class="MsoNormal">Tallahassee, FL 32306-4301<u></u><u></u></p>
<p class="MsoNormal">Phone: +1 850 645 1902<u></u><u></u></p>
<p class="MsoNormal">Fax: +1 850 644 7739<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>

</blockquote></div><br></div></div>
</blockquote></div><br></div>