We are currently planning to deploy a new HPC system with a total compute capacity exceeding 100 PF. As part of our preparation, we would like to understand which Slurm versions are considered stable and widely used at this scale.
Could you please share your recommendations or experience regarding:
1. Which Slurm version is currently running reliably on very large-scale clusters (>100 PF or >10k nodes)?
2. Whether there are any versions we should avoid due to known issues at large scale.
3. Any best practices or configuration considerations for Slurm deployments of this size.
I would take a step back and ask how you intend to install and manage this cluster.
CPU only or GPUs ? OS ? Interconnect fabric? Storage ?
Power per rack? Cooling? Monitoring?
On Sun, Nov 16, 2025, 2:39 PM KK via slurm-users < slurm-users@lists.schedmd.com> wrote:
We are currently planning to deploy a new HPC system with a total compute capacity exceeding 100 PF. As part of our preparation, we would like to understand which Slurm versions are considered stable and widely used at this scale.
Could you please share your recommendations or experience regarding:
- Which Slurm version is currently running reliably on very large-scale
clusters (>100 PF or >10k nodes)?
- Whether there are any versions we should avoid due to known issues at
large scale.
- Any best practices or configuration considerations for Slurm
deployments of this size.
-- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-leave@lists.schedmd.com
We run at about 50 PF and 1.5k nodes with about 100,000 jobs per day and we use 25.05.4, though we tend to upgrade to latest available so we will be upgrading to 25.11.* soon (when the .1 release comes out). If you are interested I'm happy to share our slurm.conf.
At least from my experience the latest releases have been stable, though you want to avoid .0 releases unless you want to be bleeding edge or need a feature. Most of the kinks are worked out by .1, definitely by .2 of any major release. There still may be weird edge cases but in general it is stable.
-Paul Edmon-
On 11/16/25 10:33 AM, John Hearns via slurm-users wrote:
I would take a step back and ask how you intend to install and manage this cluster.
CPU only or GPUs ? OS ? Interconnect fabric? Storage ?
Power per rack? Cooling? Monitoring?
On Sun, Nov 16, 2025, 2:39 PM KK via slurm-users slurm-users@lists.schedmd.com wrote:
We are currently planning to deploy a new HPC system with a total compute capacity exceeding 100 PF. As part of our preparation, we would like to understand which Slurm versions are considered stable and widely used at this scale. Could you please share your recommendations or experience regarding: 1. Which Slurm version is currently running reliably on very large-scale clusters (>100 PF or >10k nodes)? 2. Whether there are any versions we should avoid due to known issues at large scale. 3. Any best practices or configuration considerations for Slurm deployments of this size. -- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-leave@lists.schedmd.com
-
John Hearns -
KK -
Paul Edmon