Good morning,
long story: Every two years, some time after a release of Debian or Ubuntu LTS, when evaluations have ended and configurations adapted or rewritten and some reasonable maintenance date has been agreed upon by users' representatives (and directors!) and admins, we're in the situation to perform a full-upgrade (dist-upgrade) of all the machines that keep important data (while the compute nodes just get a fresh install). With the old 9-month release cycle of Slurm, in the past there was a slim chance that the upgrade from the version provided with the old release to the new release's one was small enough (since 3*9>24), but with the new 6-month cycle (4*6=24) there will always be one version too much in between. For a more or less painless transition, the manager machine running slurmdbd and slurmctld would need an intermediate version - which is not provided by Debian or Ubuntu at the time of the upgrade.
Long story shorter: Provided you run Slurm on Debian/Ubuntu: Where do you get that necessary intermediate version from? I'll have to upgrade from 22.05.8-4+deb12u3 (Bookworm) to 24.11.5-4 (Trixie) (while I'm expecting the Slurm developers to urge me to proceed to 25.11.x, I couldn't find *any* deb packages for that, plus it would immediately break Ubuntu LTS integration since Noble is at 23.11.4-1.2ubuntu5). I'd need 23.11 for Debian (Bookworm *or* Trixie) for the half-step upgrade of slurmdbd's databases, and I'd like to bump Ubuntu's version to 24.11 to eventually add our GPU machines to the cluster at a later point.
Long story short: Does someone keep those intermediate versions, and maybe builds ahead of the official distro ones? I can't be alone with this problem...
Thanks for any pointer,
Steffen
On 15/01/2026 10:43, Steffen Grunewald via slurm-users wrote:
Good morning,
[...]
Long story short: Does someone keep those intermediate versions, and maybe builds ahead of the official distro ones? I can't be alone with this problem...
Hi,
We do have the same deb-upgrade policy (we upgrade every two major Debian releases).
I simply build localy some intermediate major slurm version(s) (in /opt, for instance, from https://download.schedmd.com/slurm/), on the slurm controller, only to run the slurmdbd upgrade once. Then trash it.
It has worked well from deb9 to deb11 to deb13...
Hope this help.
Bests, Loïs
I don't run Debian or Ubuntu, rather an RHEL-based distribution, but I have needed on occasion to build intermediate versions because we tend to use Slurm from OpenHPC, and when the whole CentOS/Rocky/RHEL thing happened with RHEL8, we got stuck, and then were slow to migrate entirely so we had no path forward.
What I did is basically use the RPM specfiles from the before and after version (you can get the similar stuff for Debian), tweaked the version numbers and looked for any new config flags/dependencies that I could reasonably expect to need for the intermediate version (eg. a flag that’s in the later version that was clearly added in the intermediate version), and then built packages from that. It’s gone pretty well. It seems like a bunch of work that isn’t necessary for me to do though, agreed.
I know SchedMD was looking at binary packages, per their conversation at SC25. I hope they can consider working with OpenHPC.
-- #BlackLivesMatter ____ || \UTGERS, |---------------------------*O*--------------------------- ||_// the State | Ryan Novosielski (he/him) - novosirj@rutgers.edu || \ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus || \ of NJ | Office of Advanced Research Computing - MSB A555B, Newark `'
On Jan 15, 2026, at 04:43, Steffen Grunewald via slurm-users slurm-users@lists.schedmd.com wrote:
Good morning,
long story: Every two years, some time after a release of Debian or Ubuntu LTS, when evaluations have ended and configurations adapted or rewritten and some reasonable maintenance date has been agreed upon by users' representatives (and directors!) and admins, we're in the situation to perform a full-upgrade (dist-upgrade) of all the machines that keep important data (while the compute nodes just get a fresh install). With the old 9-month release cycle of Slurm, in the past there was a slim chance that the upgrade from the version provided with the old release to the new release's one was small enough (since 3*9>24), but with the new 6-month cycle (4*6=24) there will always be one version too much in between. For a more or less painless transition, the manager machine running slurmdbd and slurmctld would need an intermediate version - which is not provided by Debian or Ubuntu at the time of the upgrade.
Long story shorter: Provided you run Slurm on Debian/Ubuntu: Where do you get that necessary intermediate version from? I'll have to upgrade from 22.05.8-4+deb12u3 (Bookworm) to 24.11.5-4 (Trixie) (while I'm expecting the Slurm developers to urge me to proceed to 25.11.x, I couldn't find *any* deb packages for that, plus it would immediately break Ubuntu LTS integration since Noble is at 23.11.4-1.2ubuntu5). I'd need 23.11 for Debian (Bookworm *or* Trixie) for the half-step upgrade of slurmdbd's databases, and I'd like to bump Ubuntu's version to 24.11 to eventually add our GPU machines to the cluster at a later point.
Long story short: Does someone keep those intermediate versions, and maybe builds ahead of the official distro ones? I can't be alone with this problem...
Thanks for any pointer,
Steffen
-- Steffen Grunewald, Cluster Administrator Max Planck Institute for Gravitational Physics (Albert Einstein Institute) Am Mühlenberg 1 * D-14476 Potsdam-Golm * Germany ~~~ Fon: +49-331-567 7274 Mail: steffen.grunewald(at)aei.mpg.de ~~~
-- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-leave@lists.schedmd.com
I am just building/compiling with automatic CI/CD pipepline for debian11, 12 and 13 and the for amd64 and x86_64 arch (or other oses)
So we have the same slurm version everywhere.
By the way you can also build debian/ubuntu packages from the slurm source tarballs: https://slurm.schedmd.com/quickstart_admin.html#debuild
On 15/01/2026 10:43, Steffen Grunewald via slurm-users wrote:
Good morning,
long story: Every two years, some time after a release of Debian or Ubuntu LTS, when evaluations have ended and configurations adapted or rewritten and some reasonable maintenance date has been agreed upon by users' representatives (and directors!) and admins, we're in the situation to perform a full-upgrade (dist-upgrade) of all the machines that keep important data (while the compute nodes just get a fresh install). With the old 9-month release cycle of Slurm, in the past there was a slim chance that the upgrade from the version provided with the old release to the new release's one was small enough (since 3*9>24), but with the new 6-month cycle (4*6=24) there will always be one version too much in between. For a more or less painless transition, the manager machine running slurmdbd and slurmctld would need an intermediate version - which is not provided by Debian or Ubuntu at the time of the upgrade.
Long story shorter: Provided you run Slurm on Debian/Ubuntu: Where do you get that necessary intermediate version from? I'll have to upgrade from 22.05.8-4+deb12u3 (Bookworm) to 24.11.5-4 (Trixie) (while I'm expecting the Slurm developers to urge me to proceed to 25.11.x, I couldn't find *any* deb packages for that, plus it would immediately break Ubuntu LTS integration since Noble is at 23.11.4-1.2ubuntu5). I'd need 23.11 for Debian (Bookworm *or* Trixie) for the half-step upgrade of slurmdbd's databases, and I'd like to bump Ubuntu's version to 24.11 to eventually add our GPU machines to the cluster at a later point.
Long story short: Does someone keep those intermediate versions, and maybe builds ahead of the official distro ones? I can't be alone with this problem...
Thanks for any pointer,
Steffen
-
Bas van der Vlies -
Lois Taulelle -
Ryan Novosielski -
Steffen Grunewald