<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>We just use the Archive function built into slurm. That has
worked fine for us for the past 6 years. We keep 6 months of data
in the active archive.</p>
<p><br>
</p>
<p>If you have 6 years worth of data and you want to prune down to 2
years, I recommend going month by month rather than doing it in
one go. When we initially started archiving data several years
back our first pass at archiving (which at that time had 2 years
of data in it) took forever and actually caused issues with the
archive process. We worked with SchedMD, improved the archive
script built into Slurm but also decided to only archive one month
at a time which allowed it to get done in a reasonable amount of
time.</p>
<p><br>
</p>
<p>The archived data can be pulled into a different slurm database,
which is what we do for importing historic data into our XDMod
instance.<br>
</p>
<p><br>
</p>
<p>-Paul Edmon-</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 7/13/2022 4:55 PM, Timony, Mick
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:MN2PR07MB7248926A5ED0F630E07E82A9B2899@MN2PR07MB7248.namprd07.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<style type="text/css" style="display:none;">P {margin-top:0;margin-bottom:0;}</style>
<div style="font-family: Arial, Helvetica, sans-serif; font-size:
10pt; color: rgb(0, 0, 0); background-color: rgb(255, 255,
255);" class="elementToProof">
Hi Slurm Users,</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size:
10pt; color: rgb(0, 0, 0); background-color: rgb(255, 255,
255);" class="elementToProof">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size:
10pt; color: rgb(0, 0, 0); background-color: rgb(255, 255,
255);" class="elementToProof">
Currently we don't archive our SlurmDB and have 6 years' worth
of data in our SlurmDB. We are looking to start archiving our
database as it starting to get rather large, and we have decided
to keep 2 years' worth of data. I'm wondering what approaches or
scripts other groups use.<br>
<br>
The docs refer to the ArchiveScript setting at:<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size:
10pt; color: rgb(0, 0, 0); background-color: rgb(255, 255,
255);" class="elementToProof">
<a
href="https://slurm.schedmd.com/slurmdbd.conf.html#OPT_ArchiveScript"
id="LPNoLPOWALinkPreview" moz-do-not-send="true"
class="moz-txt-link-freetext">https://slurm.schedmd.com/slurmdbd.conf.html#OPT_ArchiveScript</a><br>
<br>
I've seen suggestions to import into another database that will
require keeping the schema up-to-date which seems like a
possible maintenance issue or nightmare if one forgets to update
the schema after updating Slurmdb. We also have most of the
information in an <a
href="https://slurm.schedmd.com/elasticsearch.html"
title="https://slurm.schedmd.com/elasticsearch.html"
moz-do-not-send="true">
Elasticsearch</a> instance, which will likely suite our needs
for long term historical information.</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size:
10pt; color: rgb(0, 0, 0); background-color: rgb(255, 255,
255);" class="elementToProof">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size:
10pt; color: rgb(0, 0, 0); background-color: rgb(255, 255,
255);" class="elementToProof">
<br>
</div>
<div>
<div style="font-family: Arial, Helvetica, sans-serif;
font-size: 10pt; color: rgb(0, 0, 0);" class="elementToProof">
<span style="color: rgb(0, 0, 0); background-color: rgb(255,
255, 255); display: inline !important;">What do you use to
archive this information? CSV files, SQL dumps or something
else?</span><br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif;
font-size: 10pt; color: rgb(0, 0, 0);" class="elementToProof">
<span style="color: rgb(0, 0, 0); background-color: rgb(255,
255, 255); display: inline !important;"><br>
</span></div>
<div style="font-family: Arial, Helvetica, sans-serif;
font-size: 10pt; color: rgb(0, 0, 0);" class="elementToProof">
<span style="color: rgb(0, 0, 0); background-color: rgb(255,
255, 255); display: inline !important;"><br>
</span></div>
<div style="font-family: Arial, Helvetica, sans-serif;
font-size: 10pt; color: rgb(0, 0, 0);" class="elementToProof">
<span style="color: rgb(0, 0, 0); background-color: rgb(255,
255, 255); display: inline !important;">Regards</span></div>
<div id="Signature">
<div>
<div id="divtagdefaultwrapper" dir="ltr" style="font-size:
12pt; font-family: Calibri, Arial, Helvetica, sans-serif;
color: rgb(0, 0, 0);">
<div class="BodyFragment"><font size="2">
<div class="PlainText"><span style="font-family:
arial, sans-serif; font-size: small; color:
rgb(34, 34, 34);">-- </span><br
style="font-family: arial, sans-serif; font-size:
small; color: rgb(34, 34, 34);">
<div class="gmail_signature" style="font-family:
arial, sans-serif; font-size: small; color:
rgb(34, 34, 34);">
<div dir="ltr"><span style="font-family:Helvetica">Mick
Timony</span></div>
<div dir="ltr"><span style="font-family:Helvetica"><span
style="font-family: Helvetica; font-size:
small; background-color: rgb(255, 255, 255);
display: inline !important;">Senior DevOps
Engineer</span><br>
</span><span style="font-family:Helvetica">Harvard
Medical School</span></div>
<div dir="ltr"><span style="font-family:Helvetica">--</span></div>
<div dir="ltr"><span style="font-family:Helvetica"><br>
</span></div>
</div>
</div>
</font></div>
</div>
</div>
</div>
</div>
</blockquote>
</body>
</html>