<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
h3
{mso-style-priority:9;
mso-style-link:"Heading 3 Char";
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:13.5pt;
font-family:"Calibri",sans-serif;
font-weight:bold;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.Heading3Char
{mso-style-name:"Heading 3 Char";
mso-style-priority:9;
mso-style-link:"Heading 3";
font-family:"Calibri Light",sans-serif;
color:#1F4D78;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Looking over these options it looks like Archive only happens for purged data.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Can you archive data without actually purging data? I’d like to test archives to see the output first without risking loss of content. I was thinking I could have a nightly archive copy that is up to date to
the day, but the slurmdbd can still contain a longer amount of data, maybe 3 to 6 months.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> slurm-users <slurm-users-bounces@lists.schedmd.com>
<b>On Behalf Of </b>Paul Edmon<br>
<b>Sent:</b> Friday, December 10, 2021 1:36 PM<br>
<b>To:</b> slurm-users@lists.schedmd.com<br>
<b>Subject:</b> [EXT] Re: [slurm-users] slurmdbd full backup so the primary can be purged<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div id="APLWarningText">
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" align="left">
<tbody>
<tr>
<td width="100%" style="width:100.0%;background:#E0E0E0;padding:0in 0in 0in 0in">
<p class="MsoNormal" style="mso-element:frame;mso-element-frame-hspace:2.25pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
<b><span style="color:red">APL external email warning: </span></b><span style="color:black">Verify sender
<a href="mailto:slurm-users-bounces@lists.schedmd.com">slurm-users-bounces@lists.schedmd.com</a> before clicking links or attachments</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<p> <o:p></o:p></p>
</div>
</div>
<p>Files generated by the slurmdbd archive are read back into the live database by sacctmgr. See:<o:p></o:p></p>
<h3>archive load<o:p></o:p></h3>
<p class="MsoNormal">Load in to the database previously archived data. The archive file will not be loaded if the records already exist in the database - therefore, trying to load an archive file more than once will result in an error. When this data is again
archived and purged from the database, if the old archive file is still in the directory ArchiveDir, a new archive file will be created (see ArchiveDir in the slurmdbd.conf man page), so the old file will not be overwritten and these files will have duplicate
records. <o:p></o:p></p>
<p class="MsoNormal"><i>File=</i><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">File to load into database. The specified file must exist on the slurmdbd host, which is not necessarily the machine running the command.
<o:p></o:p></p>
<p class="MsoNormal"><i>Insert=</i><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in">SQL to insert directly into the database. This should be used very cautiously since this is writing your sql into the database.
<o:p></o:p></p>
<p>So you could set up a full mirror and then read the old archives into that. You just want to make sure that mirror has archiving/purging turned off so it won't rearchive the data you restored.<o:p></o:p></p>
<p>-Paul Edmon-<o:p></o:p></p>
<div>
<p class="MsoNormal">On 12/10/2021 1:28 PM, Ransom, Geoffrey M. wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Hello<o:p></o:p></p>
<p class="MsoNormal"> Our slurmdbd database is getting rather large and affecting performance, but we want to keep usage data around for a few years for metric purposes in order to figure out how our users work. I read a suggestion to have a backup DB which
has all the usage data synced to it for metric purposes and a main slurmdbd setup for the cluster to use that cleans out old data based on your user working needs.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Is there any documentation suggesting how to set up a second slurmdbd server that will receive a copy of all the main slurmdbd entries without purging so we can start purging on the in use slurmdbd service to keep short term performance
snappy? Presumably the upgrade process will be complicated by this as well since we have to keep the archive slurmdbd setup in sync with the cluster slurmdbd.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Thanks.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">*<b>EDIT</b> before hitting send* I was re-reading the slurmdbd.conf man page and just saw the Archive* options and this sounds like it would work to implement something like this.<o:p></o:p></p>
<p class="MsoNormal">Are archive files readable by sacct and sreport, or easily manually parseable?
<o:p></o:p></p>
<p class="MsoNormal">I am going to turn these on in my test cluster, but hearing about other peoples experiences with this would probably be helpful.<o:p></o:p></p>
</blockquote>
</div>
</body>
</html>