[slurm-users] [External] [slurm 20.02.3] don't suspend nodes in down state

Florian Zillner fzillner at lenovo.com
Wed Aug 26 08:35:54 UTC 2020

Hi Herbert,

just like Angelos described, we also have logic in our poweroff script that checks if the node is really IDLE and only sends the poweroff command if that's the case.

hosts=$(scontrol show hostnames $1)
for host in $hosts; do
        scontrol show node $host | tr ' ' '\n' | grep -q 'State=IDLE+POWER$'
        if [[ $? == 1 ]]; then
                echo "node $host NOT IDLE" >>$OUTFILE
                echo "node $host IDLE" >>$OUTFILE
        ssh $host poweroff
        sleep 1


From: slurm-users <slurm-users-bounces at lists.schedmd.com> on behalf of Steininger, Herbert <herbert_steininger at psych.mpg.de>
Sent: Monday, 24 August 2020 10:52
To: Slurm User Community List <slurm-users at lists.schedmd.com>
Subject: [External] [slurm-users] [slurm 20.02.3] don't suspend nodes in down state


how can I prevent slurm, to suspend nodes, which I have set to down state for maintenance?
I know about "SuspendExcNodes", but this doesn't seem the right way, to roll out the slurm.conf every time this changes.
Is there a state that I can set so that the nodes doesn't get suspended?

It happened a few times that I was doing some stuff on a server and after our idle time (1h) slurm decided to suspend the node.


Herbert Steininger
Leiter EDV & HPC
Max-Planck-Institut für Psychiatrie
Kraepelinstr.  2-10
80804 München
Tel      +49 (0)89 / 30622-368
Mail   herbert_steininger at psych.mpg.de
Web  https://www.psych.mpg.de

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20200826/4037fdae/attachment-0001.htm>

More information about the slurm-users mailing list