<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>I've never looked at the internals of how the native Slurm
archive script works. What I can tell you is that we have never
had a problem reimporting the data back in that was dumped from
older versions into a current version database. So the import
using sacctmgr must do the conversion from the older formats to
the newer formats and handle the schema changes.</p>
<p><br>
</p>
<p>I will note that if you are storing job_scripts and envs those
can eat up a ton of space in 21.08. It looks like they've solved
that problem in 22.05 but the archive steps on 21.08 took forever
due to those scripts and envs.</p>
<p><br>
</p>
<p>-Paul Edmon-</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 7/14/2022 12:55 PM, Timony, Mick
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:MN2PR07MB72485255DD5EC73881AAC35DB2889@MN2PR07MB7248.namprd07.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<style type="text/css" style="display:none;">P {margin-top:0;margin-bottom:0;}</style>
<div style="font-family: Arial, Helvetica, sans-serif; font-size:
10pt; color: rgb(0, 0, 0); background-color: rgb(255, 255,
255);" class="elementToProof">
Hi Paul</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size:
10pt; color: rgb(0, 0, 0); background-color: rgb(255, 255,
255);" class="elementToProof">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size:
10pt; color: rgb(0, 0, 0); background-color: rgb(255, 255,
255);" class="elementToProof">
<blockquote itemscope=""
itemtype="https://schemas.microsoft.com/QuotedText"
style="border-left: 3px solid rgb(200, 200, 200);
border-top-color: rgb(200, 200, 200); border-right-color:
rgb(200, 200, 200); border-bottom-color: rgb(200, 200, 200);
padding-left: 1ex; margin-left: 0.8ex;">
<div>If you have 6 years worth of data and you want to prune
down to 2 years, I recommend going month by month rather
than doing it in one go. When we initially started
archiving data several years back our first pass at
archiving (which at that time had 2 years of data in it)
took forever and actually caused issues with the archive
process. We worked with SchedMD, improved the archive
script built into Slurm but also decided to only archive one
month at a time which allowed it to get done in a reasonable
amount of time.</div>
</blockquote>
<div class="elementToProof"><span style="font-family: Arial,
Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);
background-color: rgb(255, 255, 255);">Thanks, that is good
advice. We'd had issues with accounting in the past and had
to run slurmdb rollups which can take up to 2 weeks. It's
good to get feedback like yours. Do you what exactly the
Slurm archive script does and how it archives data or what
formats it supports?</span><br>
</div>
<div class="elementToProof"><span style="font-family: Arial,
Helvetica, sans-serif; font-size: 10pt; color: rgb(0, 0, 0);
background-color: rgb(255, 255, 255);"><br>
The docs are a little vague:<br>
<br>
<a
href="https://slurm.schedmd.com/slurmdbd.conf.html#OPT_ArchiveScript"
id="LPNoLPOWALinkPreview" moz-do-not-send="true"
class="moz-txt-link-freetext">https://slurm.schedmd.com/slurmdbd.conf.html#OPT_ArchiveScript</a><br>
<br>
"This script is used to transfer accounting records out of
the database into an archive. It is used in place of the
internal process used to archive objects. The script is
executed with no arguments, and the following environment
variables are set."<br>
<br>
</span></div>
<blockquote itemscope=""
itemtype="https://schemas.microsoft.com/QuotedText"
style="border-left: 3px solid rgb(200, 200, 200);
border-top-color: rgb(200, 200, 200); border-right-color:
rgb(200, 200, 200); border-bottom-color: rgb(200, 200, 200);
padding-left: 1ex; margin-left: 0.8ex;">
<div><br>
</div>
<div class="elementToProof">The archived data can be pulled
into a different slurm database, which is what we do for
importing historic data into our XDMod instance.</div>
</blockquote>
How do you keep track of and implement schema changes to this
database?</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size:
10pt; color: rgb(0, 0, 0); background-color: rgb(255, 255,
255);" class="elementToProof">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size:
10pt; color: rgb(0, 0, 0); background-color: rgb(255, 255,
255);" class="elementToProof">
Thanks</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size:
10pt; color: rgb(0, 0, 0); background-color: rgb(255, 255,
255);" class="elementToProof">
--Mick<br>
<br>
<br>
</div>
</blockquote>
</body>
</html>