[slurm-users] Nodes stuck in drain state

[Doug Meyer]

Could also review the node log in /varlog/slurm/ .  Often sinfo -lR will
tell you the cause, fro example mem not matching the config.

Doug

On Thu, May 25, 2023 at 5:32 AM Ole Holm Nielsen <Ole.H.Nielsen at fysik.dtu.dk>
wrote:

> On 5/25/23 13:59, Roger Mason wrote:
> > slurm 20.02.7 on FreeBSD.
>
> Uh, that's old!
>
> > I have a couple of nodes stuck in the drain state.  I have tried
> >
> > scontrol update nodename=node012 state=down reason="stuck in drain state"
> > scontrol update nodename=node012 state=resume
> >
> > without success.
> >
> > I then tried
> >
> > /usr/local/sbin/slurmctld -c
> > scontrol update nodename=node012 state=idle
> >
> > also without success.
> >
> > Is there some other method I can use to get these nodes back up?
>
> What's the output of "scontrol show node node012"?
>
> /Ole
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20230525/f0972086/attachment.htm>