[slurm-users] Munge Unauthorized credential for client UID=0 GID=0

[Andrew Lewis]

Hi

We are running a 2 node test cluster with the following configuration:

OS: Rocky 9.2
Slurm: 23.02.5
pmix-5.0.1

Jobs seem to run OK, however when they do the munge daemon always logs:
2023-10-06 15:37:41 +0200 Info:      Unauthorized credential for client UID=0 GID=0

When running munge -n tests the daemon does not log these errors.


Things we have already done:

Time is synced on the two servers using ntp. My colleague and I both checked this with chronyc sources -v:
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
^*      hidden                   2   8   377   254    +11us[  +14us] +/-   11ms

User gids\uids have been double checked, they are identical between nodes.  This occurs with any user we create.
The munge and slurm user gids\uids are also identical between nodes.
We recreated and redistributed the munge key making sure permissions are set to 400.
Restarted nodes and daemons.

Q1: Is this being caused by slurmd running as root and the job being run as a normal userid?

Q2: Is this just a cosmetic message that we can ignore?


Andy
Disclaimer - University of Cape Town This email is subject to UCT policies and email disclaimer published on our website at http://www.uct.ac.za/main/email-disclaimer or obtainable from +27 21 650 9111. If this email is not related to the business of UCT, it is sent by the sender in an individual capacity. Please report security incidents or abuse via https://csirt.uct.ac.za/report-incident
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20231007/8ca65b8c/attachment.htm>