Dear Slurm users, Keeping Slurm compute nodes up-to-date with security updates is an important task, and since security updates are now released with a high frequency, it is desirable to make the updating process as simple and automatic as possible. The tools documented here in [1] offer a possible solution. It's enlightening to browse OS errata pages such as [2] or [3]. The design used in the present approach [1] is: * Configure all compute nodes to execute a crontab job script at every reboot. This script will usually do nothing, except in case the node has a Slurm "State=down" where it will perform the configured updates. * Making updates simply consists of rebooting compute node lists with the command "scontrol reboot asap nextstate=down <node-list>". Here we have employed (abused?) the Slurm "State=down" as a *trigger* telling the update script to perform some actions. Nodes with a "Reason=Node unexpectedly rebooted" are assumed to be unrelated to the present update system and will therefore be skipped. The page [1] gives details of configuration and the optional handling of mixed-state nodes. We have tested these tools on our 700-node cluster running RockyLinux 8 and 9 for some time now, and they work reliably in our environment when updating several times per week. I would welcome any comments or bug reports regarding these tools. Best regards, Ole [1] https://github.com/OleHolmNielsen/Slurm_tools/tree/master/nodes#compute-node... [2] RockyLinux errata: https://errata.build.resf.org/ [3] AlmaLinux errata: https://errata.almalinux.org/ -- Ole Holm Nielsen PhD, Senior HPC Officer Department of Physics, Technical University of Denmark