[slurm-users] [EXT] slurm/munge problem: invalid credentials

[Olaf Gellert]

Hi Sean,

Sean Crosby wrote on 16.12.20 10:09:
> Check the firewalls between your compute node and the Slurm controller 
> to make sure that they can contact each other. Slurmctld needs to 
> contact the SlurmdPort (default 6818), and slurmd needs to contact the 
> SlurmctldPort (default 6817). Also the other compute nodes need to be 
> able to contact the new compute node on SlurmdPort.

I already did (and opened the firewalls accordingly). In the output
of tcpdump I can see that the slurmctld opens a connection to slurmd
every 100 seconds, TCP connection is established and then slurmctld
sends a data packet of 170 bytes (which probably contains the
munge stuff). The connection is afterwards closed by slurmd (sending
the FIN packet), the message "invalid credentials" appears in the
logs so this looks really like a problem of munge authentication.

Thanks for the hints though, firewalling is for sure one of the
reasons why I am quite good in using tcpdump... ;-)

Olaf

-- 
Dipl. Inform. Olaf Gellert            email  gellert at dkrz.de
Deutsches Klimarechenzentrum GmbH     phone  +49 (0)40 460094 214
Bundesstrasse 45a                     fax    +49 (0)40 460094 270
D-20146 Hamburg, Germany              www    http://www.dkrz.de

Sitz der Gesellschaft: Hamburg
Geschäftsführer: Prof. Dr. Thomas Ludwig
Registergericht: Amtsgericht Hamburg, HRB 39784