[slurm-users] SlurmDB Archive settings?

[Timony, Mick]

That's great advice. Thank you Ole.

--Mick
________________________________
From: slurm-users <slurm-users-bounces at lists.schedmd.com> on behalf of Ole Holm Nielsen <Ole.H.Nielsen at fysik.dtu.dk>
Sent: Friday, July 15, 2022 2:04 AM
To: slurm-users at lists.schedmd.com <slurm-users at lists.schedmd.com>
Subject: Re: [slurm-users] SlurmDB Archive settings?

On 7/14/22 18:49, Timony, Mick wrote:
>     Which database server and version do you run, MySQL or MariaDB?  What's
>     your Slurm version?
>
> ​mariadb 5.5.68 and a patched version of slurm 21.08.7

We run the same MariaDB on CentOS 7.9, and Slurm 21.08.8-2.

>     Did you already make appropriate database purges to reduce the size?  I
>     have some notes in my Wiki page
>     https://wiki.fysik.dtu.dk/niflheim/Slurm_database#setting-database-purge-parameters
>     <https://wiki.fysik.dtu.dk/niflheim/Slurm_database#setting-database-purge-parameters>
>
> ​No, we have not made any changes yet as am concerned that it will cause
> performance issues.

IMHO, database purging is an important way to *improve* the performance!
Reducing the database size is beneficial, also for making the daily
database dump faster.  As I wrote in the above Wiki page, you should
introduce purging very gently and decrease the purging intervals in small
steps over a number of weeks:

> A monthly purge operation can be a huge amount of work for a database depending on its size, and you certainly want to cut down the amount of work required during the purges. If you did not use purges before, it is probably a good idea to try out a series of daily purges starting with:
>
> PurgeEventAfter=2000days
> PurgeJobAfter=2000days
> PurgeResvAfter=2000days
> PurgeStepAfter=2000days
> PurgeSuspendAfter=2000days
>
> If this works well over a few days, decrease the purge interval 2000days little by little and try again (1800, 1500, etc) until you after many iterations come down to the desired final purge intervals.

Please note that it's acceptable for the slurmdbd to be slow, or even down
for many hours.  The slurmctld will cache all data (up to a certain limit)
while the slurmdbd is not responding.

/Ole


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20220715/902480c2/attachment.htm>