Generally, the troubleshooting steps which you should take for Slurm are:
squeue to look at the list of running/queued or held jobs
sinfo to show which nodes are idle, busy or down
scontrol show node to get more detailed information on a node
For problem nodes - indeed just log into any node to see what a healthy node looks like systemctl status slurmd cat /var/log/slurm/slurmd.log
On your slurm controller look at the slurmctld and slurmdbd logs
On Tue, 7 Jan 2025 at 11:30, sportlecon sportlecon via slurm-users < slurm-users@lists.schedmd.com> wrote:
slurm 24.11 - squeue displays reason "launch failed requeued held"
-- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-leave@lists.schedmd.com