[slurm-users] Munge decode failing on new node

[Ole Holm Nielsen]

You might want to check the Munge section in my Slurm Wiki page:
https://wiki.fysik.dtu.dk/niflheim/Slurm_installation#munge-authentication-service

/Ole

On 15-04-2020 19:57, Dean Schulze wrote:
> I've installed two new nodes onto my slurm cluster.  One node works, but 
> the other one complains about an invalid credential for munge.  I've 
> verified that the munge.key is the same as on all other nodes with
> 
> sudo cksum /etc/munge/munge.key
> 
> I recopied a munge.key from a node that works.  I've verified that munge 
> uid and gid are the same on the nodes.  The time is in sync on all nodes.
> 
> Here is what is in the slurmd.log:
> 
>   error: Unable to register: Unable to contact slurm controller (connect 
> failure)
>   error: Munge decode failed: Invalid credential
>   ENCODED: Wed Dec 31 17:00:00 1969
>   DECODED: Wed Dec 31 17:00:00 1969
>   error: authentication: Invalid authentication credential
>   error: slurm_receive_msg_and_forward: Protocol authentication error
>   error: service_connection: slurm_receive_msg: Protocol authentication 
> error
>   error: Unable to register: Unable to contact slurm controller (connect 
> failure)
> 
> I've checked in the munged.log and all it says is
> 
> Invalid credential