<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 1/30/23 10:35 PM, Ratnasamy, Fritz
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAARQjGPnkswLcOFxf5mD-w_dQhOBEH1GccYo_yWKuukm__67ww@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">Hi, <br>
<br>
Currently, some of our nodes are overloaded. The nhc installed
used to check the load and drain the node when it is overloaded.
However, for the past few days, it is not showing the state of
the node. When I run /usr/sbin/nhc manually, it says
<div><font face="monospace">20230130 21:25:14 [slurm]
/usr/libexec/nhc/node-mark-online <a
href="http://mcn26.chicagobooth.edu"
moz-do-not-send="true">mcn26.chicagobooth.edu</a></font></div>
<font face="monospace">/usr/libexec/nhc/node-mark-online: Not
sure how to handle node state "" on <a
href="http://mcn26.chicagobooth.edu" moz-do-not-send="true">mcn26.chicagobooth.edu</a><br>
/usr/libexec/nhc/node-mark-online: Skipping node <a
href="http://mcn26.chicagobooth.edu" moz-do-not-send="true">mcn26.chicagobooth.edu</a>
( )</font>
<div><br>
</div>
<div>It seems that it is not able to read the state of the node.
I ran <font face="monospace">scontrol show node mcn26<br>
NodeName=mcn26 Arch=x86_64 CoresPerSocket=16<br>
NodeAddr=mcn26 NodeHostName=mcn26 Version=20.11.8</font><br>
<div><br>
</div>
<div>Any idea what happened and why nhc is not reading the
state of the node anymore? <br>
</div>
</div>
</div>
</blockquote>
<p><font face="monospace"><br>
</font></p>
<p><font face="monospace">nhc is using the FQDN, slurm isn't (</font><font
face="monospace"><font face="monospace">NodeHostName=mcn26)</font>,
the query is failing. <br>
</font></p>
<p><font face="monospace">We have a line 'export HOSTNAME=$(hostname
-s)' in /etc/sysconfig/nhc</font></p>
<p><font face="monospace"><br>
</font></p>
<p><font face="monospace">-b<br>
</font></p>
<p><br>
</p>
</body>
</html>