<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>They fix this in newer versions of Slurm. We had the same issue
with older versions so we hard to run with the config_override
option on to keep the logs quiet. They changed the way logging
was done in the more recent releases and its not as chatty.</p>
<p>-Paul Edmon-<br>
</p>
<div class="moz-cite-prefix">On 5/12/22 7:35 AM, Per Lönnborg wrote:<br>
</div>
<blockquote type="cite"
cite="mid:LN2NXVUBTGU4.MZ2R42LHXEGQ2@bbweb04oc">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
Greetings,
<div><br>
</div>
<div>is there a way to lower the log rate on error messages in
slurmctld for nodes with hardware errors? </div>
<div><br>
</div>
<div>We see for example this for a node that has DIMM errors:</div>
<div><br>
</div>
<div>
<div>[2022-05-12T07:07:34.757] error: Node node37 has low
real_memory size (257642 < 257660)</div>
<div>[2022-05-12T07:07:35.760] error: Node node37 has low
real_memory size (257642 < 257660)</div>
<div>[2022-05-12T07:07:36.763] error: Node node37 has low
real_memory size (257642 < 257660)</div>
<div>[2022-05-12T07:07:37.766] error: Node node37 has low
real_memory size (257642 < 257660)</div>
<div>[2022-05-12T07:07:38.769] error: Node node37 has low
real_memory size (257642 < 257660)</div>
<div>[2022-05-12T07:07:39.773] error: Node node37 has low
real_memory size (257642 < 257660)</div>
<div>[2022-05-12T07:07:40.776] error: Node node37 has low
real_memory size (257642 < 257660)</div>
<div>[2022-05-12T07:07:41.779] error: Node node37 has low
real_memory size (257642 < 257660)</div>
<div>[2022-05-12T07:07:42.781] error: Node node37 has low
real_memory size (257642 < 257660)</div>
<div>[2022-05-12T07:07:45.143] error: Node node37 has low
real_memory size (257642 < 257660)</div>
</div>
<div><br>
</div>
<div>The log warning is correct, the node has DIMM errors, but
that´s one log entry per second. That doesn´t seem right with
such high log rate?</div>
<div><br>
</div>
<div>Thanks,</div>
<div>/ Per Lonnborg</div>
<div><br>
</div>
<div><br>
</div>
<br>
<br>
<p><font style="font-size:13.5px" size="2" face="Arial, Helvetica,
sans-serif">_______________________________________________________________<br>
Annons: Handla enkelt och smidigt hos <a
href="http://www.dpbolvw.net/click-5762941-10771045"
target="_blank" moz-do-not-send="true">Clas Ohlson</a><img
src="http://www.lduhtrp.net/image-5762941-10771045"
moz-do-not-send="true" width="1" height="1" border="0"></font>
</p>
</blockquote>
</body>
</html>