[slurm-users] SlurmDB Archive settings?

[Timony, Mick]

Hi Paul

If you have 6 years worth of data and you want to prune down to 2 years, I recommend going month by month rather than doing it in one go.  When we initially started archiving data several years back our first pass at archiving (which at that time had 2 years of data in it) took forever and actually caused issues with the archive process.  We worked with SchedMD, improved the archive script built into Slurm but also decided to only archive one month at a time which allowed it to get done in a reasonable amount of time.
Thanks, that is good advice. We'd had issues with accounting in the past and had to run slurmdb rollups which can take up to 2 weeks. It's good to get feedback like yours. Do you what exactly the Slurm archive script does and how it archives data or what formats it supports?

The docs are a little vague:

https://slurm.schedmd.com/slurmdbd.conf.html#OPT_ArchiveScript

"This script is used to transfer accounting records out of the database into an archive. It is used in place of the internal process used to archive objects. The script is executed with no arguments, and the following environment variables are set."


The archived data can be pulled into a different slurm database, which is what we do for importing historic data into our XDMod instance.
How do you keep track of and implement schema changes to this database?

Thanks
--Mick


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20220714/d0060e6b/attachment-0001.htm>