[slurm-users] Extreme long db upgrade 16.05.6 -> 17.11.3

[Kurt H Maier]

On Wed, Feb 21, 2018 at 11:56:38PM +0000, Christopher Benjamin Coffey wrote:
> Hello,
> 
> We have been trying to upgrade slurm on our cluster from 16.05.6 to 17.11.3. I'm thinking this should be doable? Past upgrades have been a breeze, and I believe during the last one, the db upgrade took like 25 minutes. Well now, the db upgrade process is taking far too long. We previously attempted the upgrade during a maintenance window and the upgrade process did not complete after 24 hrs. I gave up on the upgrade and reverted the slurm version back by restoring a backup db.

We hit this on our try as well: upgrading from 17.02.9 to 17.11.3.  We 
truncated our job history for the upgrade, and then did the rest of the 
conversion out-of-band and re-imported it after the fact.  It took us 
almost sixteen hours to convert a 1.5 million-job store.

We got hung up on precisely the same query you did, on a similarly hefty
machine.  It caused us to roll back an upgrade and try again during our
subsequent maintenance window with the above approach.

khm