[slurm-users] SlurmDB Archive settings?

[Ole Holm Nielsen]

On 7/14/22 18:49, Timony, Mick wrote:
>     Which database server and version do you run, MySQL or MariaDB?  What's
>     your Slurm version?
> 
> ​mariadb 5.5.68 and a patched version of slurm 21.08.7

We run the same MariaDB on CentOS 7.9, and Slurm 21.08.8-2.

>     Did you already make appropriate database purges to reduce the size?  I
>     have some notes in my Wiki page
>     https://wiki.fysik.dtu.dk/niflheim/Slurm_database#setting-database-purge-parameters
>     <https://wiki.fysik.dtu.dk/niflheim/Slurm_database#setting-database-purge-parameters>
> 
> ​No, we have not made any changes yet as am concerned that it will cause 
> performance issues.

IMHO, database purging is an important way to *improve* the performance! 
Reducing the database size is beneficial, also for making the daily 
database dump faster.  As I wrote in the above Wiki page, you should 
introduce purging very gently and decrease the purging intervals in small 
steps over a number of weeks:

> A monthly purge operation can be a huge amount of work for a database depending on its size, and you certainly want to cut down the amount of work required during the purges. If you did not use purges before, it is probably a good idea to try out a series of daily purges starting with:
> 
> PurgeEventAfter=2000days
> PurgeJobAfter=2000days
> PurgeResvAfter=2000days
> PurgeStepAfter=2000days
> PurgeSuspendAfter=2000days
> 
> If this works well over a few days, decrease the purge interval 2000days little by little and try again (1800, 1500, etc) until you after many iterations come down to the desired final purge intervals.

Please note that it's acceptable for the slurmdbd to be slow, or even down 
for many hours.  The slurmctld will cache all data (up to a certain limit) 
while the slurmdbd is not responding.

/Ole