<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hello Greg,</p>
<p>I have a two node set up. node1 is primary slurmctld + backup
slurmdbd and node2 is primary slurmdbd + backup slurmctld and
mysql database host.<br>
</p>
<p> My concern is if node 2 goes down, then the backup slurmdbd will
take over, then what will happen ?</p>
<p>I have read that slurmctld can cache data, but what about
slurmdbd? Not sure.<br>
</p>
<p>I have intentionally used the slurmdbd + mariadb in the second
node because I didn't want to overload the primary slurmctld.</p>
<p>I hope you all are getting the picture of how my set up is.</p>
<p>Thanks,</p>
<p>RC</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 11/1/2022 10:40 AM, Greg Wickham
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:AM6PR0202MB3270B3E34C909DE1CD5B8713AF369@AM6PR0202MB3270.eurprd02.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}div.WordSection1
{page:WordSection1;}</style>
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-US">Hi
Richard,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-US">Slurmctld
caches the updates until slurmdbd comes back online.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-US">You
can see how many records are pending for the database by
using the “sdiag” command and looking for “DBD Agent queue
size”.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-US">If
this number grows significantly it means that slurmdbd isn’t
available.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-US">
-Greg<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span
style="font-size:11.0pt">On 01/11/2022, 07:23,
"slurm-users"
<a class="moz-txt-link-rfc2396E" href="mailto:slurm-users-bounces@lists.schedmd.com"><slurm-users-bounces@lists.schedmd.com></a> wrote:<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><span
style="font-size:11.0pt"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:12.0pt;margin-left:36.0pt"><span
style="font-size:11.0pt">Hi,<br>
<br>
Just for my info, I would like to know what happens when
SlurmDBD loses <br>
connection to the backend Database, for ex, MariaDB.<br>
<br>
Does it cache the accounting info and keep them till the
DB comes back <br>
up ?, or does it panic and shut down ?<br>
<br>
Thank you,<br>
<br>
RC.<br>
<br>
<o:p></o:p></span></p>
</div>
</div>
</blockquote>
</body>
</html>