<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Going to chime in with some questions here...</p>
<p><br>
</p>
<p>Do you know how were your RPMS built? Were they built on a system
with the same packages and architecture as your nodes? That helps
(a lot). If you know the command that was used to build them and
what packages were included, that can help troubleshoot.</p>
<p>Normally it is as simple as rpmbuild -bb <downloaded slurm
source><br>
</p>
<p><br>
</p>
<p>Other notes:</p>
<p>You _must_ have a slurmctld running for slurmd to be of any use.
It will check the node named in the slurm.conf file to register
with it before starting.</p>
<p>That slurmctld _must_ be compatible with the version of slurmd
you are running.</p>
<p>slurmd should be running as root. It needs to be able to do a few
things including run the job as the user that submitted it. Things
that only root should be doing.</p>
<p><br>
</p>
<p>Brian Andrus<br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 6/2/2020 2:00 PM, Ferran Planas
Padros wrote:<br>
</div>
<blockquote type="cite"
cite="mid:c30f1f97eeff4589aad86643cdb2df9f@su.se">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text -->
<style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
<meta content="text/html; charset=UTF-8">
<style type="text/css" style="">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr"
style="font-size:12pt; color:#000000;
font-family:Calibri,Helvetica,sans-serif">
<p>Hi Ole,</p>
<p><br>
</p>
<p>I run the same version of slurm in all (master and
computing) nodes (slurm-14.03.3-2). I agree that I should
update the old nodes (which have CentOS 6.5 and 6.6) to
CentOS 7. However, it is the installation of slurm on CentOS
7.8 that is giving me all these problems, so I am skeptical
about starting the update before figuring out what is
happening.</p>
<p><br>
</p>
<p><br>
</p>
<p>Best,</p>
<p>Ferran</p>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt"
face="Calibri, sans-serif" color="#000000"><b>From:</b>
slurm-users <a class="moz-txt-link-rfc2396E" href="mailto:slurm-users-bounces@lists.schedmd.com"><slurm-users-bounces@lists.schedmd.com></a> on
behalf of Ole Holm Nielsen
<a class="moz-txt-link-rfc2396E" href="mailto:Ole.H.Nielsen@fysik.dtu.dk"><Ole.H.Nielsen@fysik.dtu.dk></a><br>
<b>Sent:</b> Tuesday, June 2, 2020 10:04:53 PM<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:slurm-users@lists.schedmd.com">slurm-users@lists.schedmd.com</a><br>
<b>Subject:</b> Re: [slurm-users] Problem with permisions.
CentOS 7.8</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">Hi Ferran,<br>
<br>
The Slurm RPMs built in the standard way will not cause any
errors with <br>
Systemd daemons. You should not have any troubles on a
correctly <br>
installed Slurm node. That is why I think you need to look
at other <br>
problems in your setup.<br>
<br>
Which versions of Slurm do you run?<br>
<br>
Which nodes run the old CentOS 6 and which Slurm versions?
You may have <br>
to upgrade to CentOS 7.<br>
<br>
Please understand that you must not mix very old Slurm
versions with new <br>
ones, see <br>
<a
href="https://wiki.fysik.dtu.dk/niflheim/Slurm_installation#upgrading-slurm"
moz-do-not-send="true">https://wiki.fysik.dtu.dk/niflheim/Slurm_installation#upgrading-slurm</a><br>
The Slurm versions may only be mixed as follows:<br>
slurmdbd >= slurmctld >= slurmd >= commands<br>
<br>
So your CentOS 7.8 compute node running slurmd must not have
a Slurm <br>
version newer than that of the slurmctld and slurmdbd nodes.<br>
<br>
/Ole<br>
<br>
<br>
On 02-06-2020 18:54, Ferran Planas Padros wrote:<br>
> However, I am setting up a calculating node, not the
master node, and <br>
> thus I have not installed slurmctld on it.<br>
> <br>
> <br>
> After some digging, I have found that all these files:<br>
> <br>
> /run/systemd/generator.late/slurm.service<br>
> <br>
>
/run/systemd/generator.late/runlevel5.target.wants/slurm.service<br>
> <br>
>
/run/systemd/generator.late/runlevel4.target.wants/slurm.service<br>
> <br>
>
/run/systemd/generator.late/runlevel3.target.wants/slurm.service<br>
> <br>
>
/run/systemd/generator.late/runlevel2.target.wants/slurm.service<br>
> <br>
> <br>
> Which are a copy of each other and are generated by <br>
> systemd-sysv-generator, point to the slurmctld.pid, not
to the slurm.pid<br>
> <br>
> <br>
> [Unit]<br>
> <br>
> Documentation=man:systemd-sysv-generator(8)<br>
> <br>
> SourcePath=/etc/rc.d/init.d/slurm<br>
> <br>
> Description=LSB: slurm daemon management<br>
> <br>
> Before=runlevel2.target<br>
> <br>
> Before=runlevel3.target<br>
> <br>
> Before=runlevel4.target<br>
> <br>
> Before=runlevel5.target<br>
> <br>
> Before=shutdown.target<br>
> <br>
> After=remote-fs.target<br>
> <br>
> After=network-online.target<br>
> <br>
> After=munge.service<br>
> <br>
> After=nss-lookup.target<br>
> <br>
> After=network-online.target<br>
> <br>
> Wants=network-online.target<br>
> <br>
> Conflicts=shutdown.target<br>
> <br>
> <br>
> [Service]<br>
> <br>
> Type=forking<br>
> <br>
> Restart=no<br>
> <br>
> TimeoutSec=5min<br>
> <br>
> IgnoreSIGPIPE=no<br>
> <br>
> KillMode=process<br>
> <br>
> GuessMainPID=no<br>
> <br>
> RemainAfterExit=no<br>
> <br>
> *PIDFile=/var/run/slurmctld.pid*<br>
> <br>
> ExecStart=/etc/rc.d/init.d/slurm start<br>
> <br>
> ExecStop=/etc/rc.d/init.d/slurm stop<br>
> <br>
> ~<br>
> <br>
> <br>
> <br>
> How can I make it to avoid this? Besides editing the
files manually, <br>
> which will go back to the original after reboot.<br>
> <br>
> <br>
> Thanks,<br>
> <br>
> Ferran<br>
> <br>
> <br>
>
------------------------------------------------------------------------<br>
> *From:* slurm-users
<a class="moz-txt-link-rfc2396E" href="mailto:slurm-users-bounces@lists.schedmd.com"><slurm-users-bounces@lists.schedmd.com></a> on behalf of <br>
> Rodrigo Santibáñez <a class="moz-txt-link-rfc2396E" href="mailto:rsantibanez.uchile@gmail.com"><rsantibanez.uchile@gmail.com></a><br>
> *Sent:* Tuesday, June 2, 2020 6:40:48 PM<br>
> *To:* Slurm User Community List<br>
> *Subject:* Re: [slurm-users] Problem with permisions.
CentOS 7.8<br>
> Yes, you have both daemons, installed with the slurm
rpm.The slurmd (all <br>
> nodes) communicates with slurmctld (runs in the main
master node and, <br>
> optionally, in a backup node).<br>
> <br>
> You do not need to run slurmd as the slurm user. Use
`systemctld enable <br>
> slurmctld` (and slurmd) followed by `systemclt start
slurmctld`. Use <br>
> restart instead of start if you change the
configuration only if `sudo <br>
> scontrol reconfigure` asks for it.<br>
> <br>
> If you run as root `slurmctld -Dvvvv` and `slurmd
-Dvvvv` you'll see <br>
> debug outputs to see further problems with
configuration. The slurmd <br>
> needs slurmctld running or will output "error: Unable
to register: <br>
> Unable to contact slurm controller (connect failure)"<br>
> <br>
> You should find the services here:<br>
> -rw-r--r-- 1 root root 339 may 30 20:18 <br>
> /usr/lib/systemd/system/slurmctld.service<br>
> -rw-r--r-- 1 root root 342 may 30 20:18 <br>
> /usr/lib/systemd/system/slurmdbd.service<br>
> -rw-r--r-- 1 root root 398 may 30 20:18 <br>
> /usr/lib/systemd/system/slurmd.service<br>
> <br>
> Feel free to ask for more information,<br>
> Best regards<br>
> <br>
> El mar., 2 jun. 2020 a las 11:12, Ferran Planas Padros
<br>
> (<<a class="moz-txt-link-abbreviated" href="mailto:ferran.padros@su.se">ferran.padros@su.se</a> <<a
href="mailto:ferran.padros@su.se" moz-do-not-send="true">mailto:ferran.padros@su.se</a>>>)
escribió:<br>
> <br>
> <br>
> Hi Ole,<br>
> <br>
> <br>
> Thanks for your answer and your time. I'd
appreciate if you, or<br>
> someone else, could make a final look at my case.<br>
> <br>
> After your suggestions and comments, I have re-done
the whole<br>
> installation for Munge and Slurm. I uninstalled and
remoced all<br>
> previous rpms and restarted from scratch. Munge
works with no<br>
> problem, however it does not happen the same with
slurm (for which I<br>
> have used the instructions given in the link you
attached)<br>
> <br>
> <br>
> - If I run /usr/bin/slurmd -D vvvvv as root user, I
get the verbose<br>
> until the line 'slurmd: debug2: No acct_gather.conf
file<br>
> (/etc/slurm/acct_gather.conf)' where the verbose
stops. After I do<br>
> Ctrl+C, I get<br>
> <br>
> <br>
> slurmd: all threads complete<br>
> <br>
> slurmd: Consumable Resources (CR) Node Selection
plugin shutting<br>
> down ...<br>
> <br>
> slurmd: Munge cryptographic signature plugin
unloaded<br>
> <br>
> slurmd: Slurmd shutdown completing<br>
> <br>
> <br>
> - After that, if I run 'systemctl start slurmd' and
'systemctl<br>
> status slurmd', also as root user, I get:<br>
> <br>
> *●*slurmd.service - Slurm node daemon<br>
> <br>
> Loaded: loaded (/etc/systemd/system/slurmd.service;
enabled; vendor<br>
> preset: disabled)<br>
> <br>
> Active: *active (running)*since Tue 2020-06-02
16:53:51 CEST; 33s ago<br>
> <br>
> Process: 2750 ExecStart=/usr/sbin/slurmd -d
/usr/sbin/slurmstepd<br>
> $SLURMD_OPTIONS (code=exited, status=0/SUCCESS)<br>
> <br>
> Main PID: 2752 (slurmd)<br>
> <br>
> CGroup: /system.slice/slurmd.service<br>
> <br>
> └─2752 /usr/sbin/slurmd -d /usr/sbin/slurmstepd<br>
> <br>
> <br>
> Jun 02 16:53:51 roos21.organ.su.se <<a
href="http://roos21.organ.su.se" moz-do-not-send="true">http://roos21.organ.su.se</a>><br>
> systemd[1]: Starting Slurm node daemon...<br>
> <br>
> Jun 02 16:53:51 roos21.organ.su.se <<a
href="http://roos21.organ.su.se" moz-do-not-send="true">http://roos21.organ.su.se</a>><br>
> systemd[1]: Can't open PID file
/var/run/slurm/slurmd.pid (yet?)<br>
> after start: No such file or directory<br>
> <br>
> Jun 02 16:53:51 roos21.organ.su.se <<a
href="http://roos21.organ.su.se" moz-do-not-send="true">http://roos21.organ.su.se</a>><br>
> systemd[1]: Started Slurm node daemon.<br>
> <br>
> <br>
> - Next, I kill the slurmd process, and I run, as
slurm user,<br>
> 'systemctl start slurm'. Which does not work and
returns the<br>
> following in the journalctl -xe:<br>
> <br>
> <br>
> Jun 02 16:56:01 roos21.organ.su.se <<a
href="http://roos21.organ.su.se" moz-do-not-send="true">http://roos21.organ.su.se</a>><br>
> systemd[1]: Starting LSB: slurm daemon
management...<br>
> <br>
> -- Subject: Unit slurm.service has begun start-up<br>
> <br>
> -- Defined-By: systemd<br>
> <br>
> -- Support: <a
href="http://lists.freedesktop.org/mailman/listinfo/systemd-devel"
moz-do-not-send="true">
http://lists.freedesktop.org/mailman/listinfo/systemd-devel</a><br>
> <br>
> --<br>
> <br>
> -- Unit slurm.service has begun starting up.<br>
> <br>
> Jun 02 16:56:01 roos21.organ.su.se <<a
href="http://roos21.organ.su.se" moz-do-not-send="true">http://roos21.organ.su.se</a>><br>
> slurm[2805]: starting slurmd: [OK]<br>
> <br>
> Jun 02 16:56:01 roos21.organ.su.se <<a
href="http://roos21.organ.su.se" moz-do-not-send="true">http://roos21.organ.su.se</a>><br>
> systemd[1]: Can't open PID file
/var/run/slurmctld.pid (yet?) after<br>
> start: No such file or directory<br>
> <br>
> Jun 02 16:56:37 roos21.organ.su.se <<a
href="http://roos21.organ.su.se" moz-do-not-send="true">http://roos21.organ.su.se</a>><br>
> polkitd[1316]: *Unregistered Authentication Agent
for<br>
> unix-process:2792:334647 (system bus name :1.46,
object path<br>
> /org/freedesktop*<br>
> <br>
> Jun 02 16:56:38 roos21.organ.su.se <<a
href="http://roos21.organ.su.se" moz-do-not-send="true">http://roos21.organ.su.se</a>><br>
> sudo[2790]: pam_unix(sudo:session): session closed
for user slurm<br>
> <br>
> <br>
> Something that I don't really understand because I
have not<br>
> installed slurmctld. The slurmctld.service file
does not even exist.<br>
> <br>
> <br>
> Any idea?<br>
> <br>
> <br>
> Many thanks,<br>
> <br>
> Ferran<br>
> <br>
> <br>
> <br>
>
------------------------------------------------------------------------<br>
> *From:* slurm-users
<<a class="moz-txt-link-abbreviated" href="mailto:slurm-users-bounces@lists.schedmd.com">slurm-users-bounces@lists.schedmd.com</a><br>
> <<a
href="mailto:slurm-users-bounces@lists.schedmd.com"
moz-do-not-send="true">mailto:slurm-users-bounces@lists.schedmd.com</a>>>
on behalf of Ole<br>
> Holm Nielsen <<a class="moz-txt-link-abbreviated" href="mailto:Ole.H.Nielsen@fysik.dtu.dk">Ole.H.Nielsen@fysik.dtu.dk</a><br>
> <<a href="mailto:Ole.H.Nielsen@fysik.dtu.dk"
moz-do-not-send="true">mailto:Ole.H.Nielsen@fysik.dtu.dk</a>>><br>
> *Sent:* Tuesday, June 2, 2020 12:03:27 PM<br>
> *To:* Slurm User Community List<br>
> *Subject:* Re: [slurm-users] Problem with
permisions. CentOS 7.8<br>
> Hi Ferran,<br>
> <br>
> Please install Slurm software in the standard way,
see<br>
> <a
href="https://wiki.fysik.dtu.dk/niflheim/Slurm_installation"
moz-do-not-send="true">https://wiki.fysik.dtu.dk/niflheim/Slurm_installation</a><br>
> <br>
> It seems that you have some unusual way to manage
your Linux<br>
> systems. In<br>
> Stockholm and Sweden there are many Slurm experts
at the HPC centers<br>
> which<br>
> might be able to help you more directly.<br>
> <br>
> Best regards,<br>
> Ole<br>
> <br>
> On 6/2/20 11:58 AM, Ferran Planas Padros wrote:<br>
> > I did a fresh installation with the EPEL repo,
and installing munge from <br>
> > it and it worked. To have the slurm user for
munge was definitely a <br>
> > problem, but that is the set up we have on the
CentOS 6. Now I've learnt <br>
> > my lesson for future installations, thanks to
everyone!<br>
> > <br>
> > <br>
> > Now, I have a follow up question, if you don't
mind. I am now trying to <br>
> > run slurm, and it crashes:<br>
> > <br>
> > <br>
> > [root@roos21 ~]# systemctl status
slurm.service<br>
> > <br>
> > *●*slurm.service - LSB: slurm daemon
management<br>
> > <br>
> > Loaded: loaded (/etc/rc.d/init.d/slurm; bad;
vendor preset: disabled)<br>
> > <br>
> > Active: *failed*(Result: protocol) since Tue
2020-06-02 11:45:33 CEST; <br>
> > 3min 33s ago<br>
> > <br>
> > Docs: man:systemd-sysv-generator(8)<br>
> > <br>
> > <br>
> > Jun 02 11:45:33 roos21.organ.su.se <<a
href="http://roos21.organ.su.se" moz-do-not-send="true">http://roos21.organ.su.se</a>>
systemd[1]: Starting<br>
> LSB: slurm daemon<br>
> > management...<br>
> > <br>
> > Jun 02 11:45:33 roos21.organ.su.se <<a
href="http://roos21.organ.su.se" moz-do-not-send="true">http://roos21.organ.su.se</a>>
slurm[18223]:<br>
> starting slurmd: [OK]<br>
> > <br>
> > Jun 02 11:45:33 roos21.organ.su.se <<a
href="http://roos21.organ.su.se" moz-do-not-send="true">http://roos21.organ.su.se</a>>
systemd[1]: Can't<br>
> open PID file<br>
> > /var/run/slurmctld.pid (yet?) after start: No
such file or directory<br>
> > <br>
> > Jun 02 11:45:33 roos21.organ.su.se <<a
href="http://roos21.organ.su.se" moz-do-not-send="true">http://roos21.organ.su.se</a>>
systemd[1]: *Failed<br>
> to start LSB: slurm<br>
> > daemon management.*<br>
> > <br>
> > Jun 02 11:45:33 roos21.organ.su.se <<a
href="http://roos21.organ.su.se" moz-do-not-send="true">http://roos21.organ.su.se</a>>
systemd[1]: *Unit<br>
> slurm.service entered<br>
> > failed state.*<br>
> > <br>
> > Jun 02 11:45:33 roos21.organ.su.se <<a
href="http://roos21.organ.su.se" moz-do-not-send="true">http://roos21.organ.su.se</a>>
systemd[1]:<br>
> *slurm.service failed.*<br>
> > <br>
> > <br>
> > <br>
> > The thing is that this is a computing node,
not the master node, so <br>
> > slurmctld is not installed. Why do I get this
error?<br>
> > <br>
> > <br>
> > Many thanks, and my apologies for this rather
simple questions. I am a <br>
> > newbie on this.<br>
> > <br>
> > <br>
> > Best,<br>
> > <br>
> > Ferran<br>
> > <br>
> >
--------------------------------------------------------------------------<br>
> > *From:* slurm-users
<<a class="moz-txt-link-abbreviated" href="mailto:slurm-users-bounces@lists.schedmd.com">slurm-users-bounces@lists.schedmd.com</a><br>
> <<a
href="mailto:slurm-users-bounces@lists.schedmd.com"
moz-do-not-send="true">mailto:slurm-users-bounces@lists.schedmd.com</a>>>
on behalf of<br>
> > Renata Maria Dart <<a class="moz-txt-link-abbreviated" href="mailto:renata@slac.stanford.edu">renata@slac.stanford.edu</a>
<<a href="mailto:renata@slac.stanford.edu"
moz-do-not-send="true">mailto:renata@slac.stanford.edu</a>>><br>
> > *Sent:* Friday, May 29, 2020 6:33:58 PM<br>
> > *To:* <a class="moz-txt-link-abbreviated" href="mailto:Ole.H.Nielsen@fysik.dtu.dk">Ole.H.Nielsen@fysik.dtu.dk</a> <<a
href="mailto:Ole.H.Nielsen@fysik.dtu.dk"
moz-do-not-send="true">mailto:Ole.H.Nielsen@fysik.dtu.dk</a>>;<br>
> Slurm User Community List<br>
> > *Subject:* Re: [slurm-users] Problem with
permisions. CentOS 7.8<br>
> > Hi, don't know if this might be your problem
but I ran into an issue<br>
> > on centos 7.8 where /var/run/munge was not
being created at boottime<br>
> > because I didn't have the munge user in the
local password file. I<br>
> > have the munge user in AD and once the system
is up I can start munge<br>
> > successfully, but AD wasn't available early
enough during boot for the<br>
> > munge startup to see it. I added these lines
to the munge systemctl<br>
> > file:<br>
> > <br>
> > PermissionsStartOnly=true<br>
> > ExecStartPre=-/usr/bin/mkdir -m 0755 -p
/var/run/munge<br>
> > ExecStartPre=-/usr/bin/chown -R munge:munge
/var/run/munge<br>
> > <br>
> > and my system now starts munge up fine during
a reboot.<br>
> > <br>
> > Renata<br>
> > <br>
> > On Fri, 29 May 2020, Ole Holm Nielsen wrote:<br>
> > <br>
> >> Hi Ferran,<br>
> >><br>
> >> When you have a CentOS 7 system with the
EPEL repo enabled, and you have<br>
> >> installed the munge RPM from EPEL, then
things should be working correctly.<br>
> >><br>
> >> Since systemctl tells you that Munge
service didn't start correctly, then it<br>
> >> seems to me that you have a problem in the
general configuration of your CentOS<br>
> >> 7 system. You should check
/var/log/messages and "journalctl -xe" for munge<br>
> >> errors. It is really hard for other
people to guess what may be wrong in your<br>
> >> system.<br>
> >><br>
> >> My 2 cents worth: Maybe you could make a
fresh CentOS 7.8 installation on a<br>
> >> test system and install the Munge service
(and nothing else) according to<br>
> >> instructions in <a
href="https://wiki.fysik.dtu.dk/niflheim/Slurm_installation"
moz-do-not-send="true">
https://wiki.fysik.dtu.dk/niflheim/Slurm_installation</a>.
This<br>
> >> *really* has got to work!<br>
> >><br>
> >> /Ole<br>
> >><br>
> >><br>
> >> On 29-05-2020 10:23, Ferran Planas Padros
wrote:<br>
> >>> Hello everyone,<br>
> >>><br>
> >>><br>
> >>> Here it comes everything I've done.<br>
> >>><br>
> >>><br>
> >>> - About Ole's answer:<br>
> >>><br>
> >>> Yes, we have slurm as the user to
control munge. Following your comment, I<br>
> >>> have changed the ownership of the
munge files and tried to start munge as<br>
> >>> munge user. However, it also failed.<br>
> >>><br>
> >>> Also, I first installed munge from a
repository. I've seen your suggestion of<br>
> >>> installing from EPEL. So I uninstalled
and installed again. Same result<br>
> >>><br>
> >>> - About SELinux: It is disables<br>
> >>><br>
> >>> - The output of ps -ef | grep munge
is:<br>
> >>><br>
> >>><br>
> >>> root534051530 10:18 pts/000:00:00 grep
--color=auto *munge*<br>
> >>><br>
> >>><br>
> >>> - The outputs of munge -n is:<br>
> >>><br>
> >>><br>
> >>> Failed to access
"/var/run/munge/munge.socket.2": No such file or directory<br>
> >>><br>
> >>><br>
> >>> - Same for unmunge<br>
> >>><br>
> >>><br>
> >>> - Output for sudo systemctl status
--full munge<br>
> >>><br>
> >>><br>
> >>> *?*munge.service - MUNGE
authentication service<br>
> >>><br>
> >>> Loaded: loaded
(/usr/lib/systemd/system/munge.service; enabled; vendor
preset:<br>
> >>> disabled)<br>
> >>><br>
> >>> Active: *failed*(Result: exit-code)
since Fri 2020-05-29 10:15:52 CEST; 4min<br>
> >>> 18s ago<br>
> >>><br>
> >>> Docs: man:munged(8)<br>
> >>><br>
> >>> Process: 5333
ExecStart=/usr/sbin/munged *(code=exited, status=1/FAILURE)*<br>
> >>><br>
> >>><br>
> >>> May 29 10:15:52 roos21.organ.su.se
<<a href="http://roos21.organ.su.se"
moz-do-not-send="true">http://roos21.organ.su.se</a>>
systemd[1]: Starting<br>
> MUNGE authentication<br>
> >>> service...<br>
> >>><br>
> >>> May 29 10:15:52 roos21.organ.su.se
<<a href="http://roos21.organ.su.se"
moz-do-not-send="true">http://roos21.organ.su.se</a>>
systemd[1]:<br>
> *munge.service: control process<br>
> >>> exited, code=exited status=1*<br>
> >>><br>
> >>> May 29 10:15:52 roos21.organ.su.se
<<a href="http://roos21.organ.su.se"
moz-do-not-send="true">http://roos21.organ.su.se</a>>
systemd[1]: *Failed<br>
> to start MUNGE<br>
> >>> authentication service.*<br>
> >>><br>
> >>> May 29 10:15:52 roos21.organ.su.se
<<a href="http://roos21.organ.su.se"
moz-do-not-send="true">http://roos21.organ.su.se</a>>
systemd[1]: *Unit<br>
> munge.service entered<br>
> >>> failed state.*<br>
> >>><br>
> >>> May 29 10:15:52 roos21.organ.su.se
<<a href="http://roos21.organ.su.se"
moz-do-not-send="true">http://roos21.organ.su.se</a>>
systemd[1]:<br>
> *munge.service failed.*<br>
> >>><br>
> >>><br>
> >>> - Regarding NTP, I get this message:<br>
> >>><br>
> >>><br>
> >>> Unable to talk to NTP daemon. Is it
running?<br>
> >>><br>
> >>><br>
> >>> It is the same message I get in the
nodes that DO work. All nodes are sync in<br>
> >>> time and date with the central node<br>
> >>><br>
> >>><br>
> >>>
------------------------------------------------------------------------<br>
> >>> *From:* slurm-users
<<a class="moz-txt-link-abbreviated" href="mailto:slurm-users-bounces@lists.schedmd.com">slurm-users-bounces@lists.schedmd.com</a><br>
> <<a
href="mailto:slurm-users-bounces@lists.schedmd.com"
moz-do-not-send="true">mailto:slurm-users-bounces@lists.schedmd.com</a>>>
on behalf of Ole<br>
> >>> Holm Nielsen
<<a class="moz-txt-link-abbreviated" href="mailto:Ole.H.Nielsen@fysik.dtu.dk">Ole.H.Nielsen@fysik.dtu.dk</a> <<a
href="mailto:Ole.H.Nielsen@fysik.dtu.dk"
moz-do-not-send="true">mailto:Ole.H.Nielsen@fysik.dtu.dk</a>>><br>
> >>> *Sent:* Friday, May 29, 2020 9:56:10
AM<br>
> >>> *To:* <a class="moz-txt-link-abbreviated" href="mailto:slurm-users@lists.schedmd.com">slurm-users@lists.schedmd.com</a>
<<a href="mailto:slurm-users@lists.schedmd.com"
moz-do-not-send="true">mailto:slurm-users@lists.schedmd.com</a>><br>
> >>> *Subject:* Re: [slurm-users] Problem
with permisions. CentOS 7.8<br>
> >>> On 29-05-2020 08:46, Sudeep Narayan
Banerjee wrote:<br>
> >>>> also check:<br>
> >>>> a) whether NTP has been setup and
communicating with master node<br>
> >>>> b) iptables may be flushed
(iptables -L)<br>
> >>>> c) SeLinux to disabled, to check :<br>
> >>>> getenforce<br>
> >>>> vim /etc/sysconfig/selinux<br>
> >>>> (change SELINUX=enforcing to
SELINUX=disabled and save the file and reboot)<br>
> >>><br>
> >>> There is no reason to disable SELinux
for running the Munge service.<br>
> >>> It's a pretty bad idea to lower the
security just for the sake of<br>
> >>> convenience!<br>
> >>><br>
> >>> /Ole<br>
> >>><br>
> >>><br>
> >>>> On Fri, May 29, 2020 at 12:08 PM
Sudeep Narayan Banerjee<br>
> >>>> <<a class="moz-txt-link-abbreviated" href="mailto:snbanerjee@iitgn.ac.in">snbanerjee@iitgn.ac.in</a> <<a
href="mailto:snbanerjee@iitgn.ac.in"
moz-do-not-send="true">mailto:snbanerjee@iitgn.ac.in</a>><br>
> <<a href="mailto:snbanerjee@iitgn.ac.in"
moz-do-not-send="true">mailto:snbanerjee@iitgn.ac.in</a>>>
wrote:<br>
> >>>><br>
> >>>> I have not checked on the
CentOS7.8<br>
> >>>> a) if /var/run/munge folder
does not exist then please double check<br>
> >>>> whether munge has been
installed or not<br>
> >>>> b) user root or sudo user to
do<br>
> >>>> ps -ef | grep munge<br>
> >>>> kill -9 <PID> //where
PID is the Process ID for munge (if the<br>
> >>>> process is running at all);
else<br>
> >>>><br>
> >>>> which munged<br>
> >>>> /etc/init.d/munge start<br>
> >>>><br>
> >>>> please let me know the the
output of:<br>
> >>>><br>
> >>>> |$ munge -n|<br>
> >>>><br>
> >>>> |$ munge -n | unmunge|<br>
> >>>><br>
> >>>> |$ sudo systemctl status
--full munge<br>
> >>>><br>
> >>>> |<br>
> >>>><br>
> >>>> Thanks & Regards,<br>
> >>>> Sudeep Narayan Banerjee<br>
> >>>> System Analyst | Scientist B<br>
> >>>> Indian Institute of
Technology Gandhinagar<br>
> >>>> Gujarat, INDIA<br>
> >>>><br>
> >>>><br>
> >>>> On Fri, May 29, 2020 at 11:55
AM Bjørn-Helge Mevik<br>
> >>>> <<a class="moz-txt-link-abbreviated" href="mailto:b.h.mevik@usit.uio.no">b.h.mevik@usit.uio.no</a>
<<a href="mailto:b.h.mevik@usit.uio.no"
moz-do-not-send="true">mailto:b.h.mevik@usit.uio.no</a>><br>
> <<a href="mailto:b.h.mevik@usit.uio.no"
moz-do-not-send="true">mailto:b.h.mevik@usit.uio.no</a>>>
wrote:<br>
> >>>><br>
> >>>> Ferran Planas Padros
<<a class="moz-txt-link-abbreviated" href="mailto:ferran.padros@su.se">ferran.padros@su.se</a> <<a
href="mailto:ferran.padros@su.se" moz-do-not-send="true">mailto:ferran.padros@su.se</a>><br>
> >>>> <<a
href="mailto:ferran.padros@su.se" moz-do-not-send="true">mailto:ferran.padros@su.se</a>>>
writes:<br>
> >>>><br>
> >>>> > I run the command
as slurm user, and the /var/log/munge<br>
> >>>> folder does belong to
slurm.<br>
> >>>><br>
> >>>> For security reasons, I
strongly advise that you run munged as a<br>
> >>>> separate user, which is
unprivileged and not used for anything else.<br>
> >>>><br>
> >>>> -- Regards,<br>
> >>>> Bjørn-Helge Mevik, dr.
scient,<br>
> >>>> Department for Research
Computing, University of Oslo<br>
> <br>
<br>
<br>
</div>
</span></font>
</blockquote>
</body>
</html>