[slurm-users] Slurm Upgrade

[Paul Edmon]

We have hit this when we naively ran using the service and it timed out 
and borked the database.  Fortunately we had a backup to go back to.  
Since then we have run it straight from the command line.  Like yours 
our production DB is now 23 GB for 6 months worth of data so major 
schema updates take roughly 1-2 hours for us.

-Paul Edmon-

On 11/2/2020 11:15 AM, Chris Samuel wrote:
> On 11/2/20 7:31 am, Paul Edmon wrote:
>
>> e. Run slurmdbd -Dvvvvv to do the database upgrade. Depending on the 
>> upgrade this can take a while because of database schema changes.
>
> I'd like to emphasis the importance of doing the DB upgrade in this 
> way, do not use systemctl for this as if systemd runs out of patience 
> waiting for slurmdbd to finish the migration and start up it can kill 
> it part way through the migration.
>
> Fortunately not something I've run into myself, but as our mysqldump 
> of our production DB is approaching 100GB now it's not something we 
> want to run into!
>
> All the best,
> Chris