<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Files generated by the slurmdbd archive are read back into the
live database by sacctmgr. See:</p>
<h3>archive load</h3>
Load in to the database previously archived data. The archive file
will not be
loaded if the records already exist in the database - therefore,
trying to load
an archive file more than once will result in an error. When this
data is again
archived and purged from the database, if the old archive file is
still in the
directory ArchiveDir, a new archive file will be created (see
ArchiveDir in the
slurmdbd.conf man page), so the old file will not be overwritten and
these files
will have duplicate records.
<p>
</p>
<dl compact="compact">
<dt><i>File=</i></dt>
<dd>File to load into database. The specified file must exist on
the slurmdbd host,
which is not necessarily the machine running the command.
</dd>
<dt><i>Insert=</i></dt>
<dd>SQL to insert directly into the database. This should be used
very
cautiously since this is writing your sql into the database.
</dd>
</dl>
<p>So you could set up a full mirror and then read the old archives
into that. You just want to make sure that mirror has
archiving/purging turned off so it won't rearchive the data you
restored.<br>
</p>
<p>-Paul Edmon-<br>
</p>
<div class="moz-cite-prefix">On 12/10/2021 1:28 PM, Ransom, Geoffrey
M. wrote:<br>
</div>
<blockquote type="cite"
cite="mid:b09894dad9294c2c9661ad4134e05c54@jhuapl.edu">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}div.WordSection1
{page:WordSection1;}</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hello<o:p></o:p></p>
<p class="MsoNormal"> Our slurmdbd database is getting rather
large and affecting performance, but we want to keep usage
data around for a few years for metric purposes in order to
figure out how our users work. I read a suggestion to have a
backup DB which has all the usage data synced to it for metric
purposes and a main slurmdbd setup for the cluster to use that
cleans out old data based on your user working needs.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Is there any documentation suggesting how
to set up a second slurmdbd server that will receive a copy of
all the main slurmdbd entries without purging so we can start
purging on the in use slurmdbd service to keep short term
performance snappy? Presumably the upgrade process will be
complicated by this as well since we have to keep the archive
slurmdbd setup in sync with the cluster slurmdbd.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">*<b>EDIT</b> before hitting send* I was
re-reading the slurmdbd.conf man page and just saw the
Archive* options and this sounds like it would work to
implement something like this.<o:p></o:p></p>
<p class="MsoNormal">Are archive files readable by sacct and
sreport, or easily manually parseable?
<o:p></o:p></p>
<p class="MsoNormal">I am going to turn these on in my test
cluster, but hearing about other peoples experiences with this
would probably be helpful.<o:p></o:p></p>
</div>
</blockquote>
</body>
</html>