[slurm-users] SlurmDB Archive settings?
pedmon at cfa.harvard.edu
Thu Jul 14 13:10:58 UTC 2022
We just use the Archive function built into slurm. That has worked fine
for us for the past 6 years. We keep 6 months of data in the active
If you have 6 years worth of data and you want to prune down to 2 years,
I recommend going month by month rather than doing it in one go. When
we initially started archiving data several years back our first pass at
archiving (which at that time had 2 years of data in it) took forever
and actually caused issues with the archive process. We worked with
SchedMD, improved the archive script built into Slurm but also decided
to only archive one month at a time which allowed it to get done in a
reasonable amount of time.
The archived data can be pulled into a different slurm database, which
is what we do for importing historic data into our XDMod instance.
On 7/13/2022 4:55 PM, Timony, Mick wrote:
> Hi Slurm Users,
> Currently we don't archive our SlurmDB and have 6 years' worth of data
> in our SlurmDB. We are looking to start archiving our database as it
> starting to get rather large, and we have decided to keep 2 years'
> worth of data. I'm wondering what approaches or scripts other groups use.
> The docs refer to the ArchiveScript setting at:
> I've seen suggestions to import into another database that will
> require keeping the schema up-to-date which seems like a possible
> maintenance issue or nightmare if one forgets to update the schema
> after updating Slurmdb. We also have most of the information in an
> Elasticsearch <https://slurm.schedmd.com/elasticsearch.html> instance,
> which will likely suite our needs for long term historical information.
> What do you use to archive this information? CSV files, SQL dumps or
> something else?
> Mick Timony
> Senior DevOps Engineer
> Harvard Medical School
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the slurm-users