<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Chiming in on Michael's suggestion.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
You can specify the same hostname in the slurm.conf but for the on-premise nodes you either set the DNS or the /etc/hosts entry to the local (=private) IP address.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
For the cloud nodes you set DNS or the hosts entry to the publicly reachable IP.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
example <span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;">
/etc/hosts </span>on-prem node:</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
10.10.10.10 <span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;">myslurmcontroller<span> </span></span></div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;">example
<span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;">
/etc/hosts </span>cloud node:</span><br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;">50.50.50.50 <span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;">myslurmcontroller<span> </span></span></span></div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;"><br>
</span></div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;">example slurm.conf for both locations</span></div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;">ControlMachine=<span style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); display: inline !important;">myslurmcontroller<span> </span></span><br>
</span></div>
<div>
<div id="Signature">
<div>
<div style="">
<div style=""></div>
</div>
</div>
</div>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> slurm-users <slurm-users-bounces@lists.schedmd.com> on behalf of Sajesh Singh <ssingh@amnh.org><br>
<b>Sent:</b> Friday, 22 January 2021 22:17<br>
<b>To:</b> Slurm User Community List <slurm-users@lists.schedmd.com><br>
<b>Subject:</b> [External] Re: [slurm-users] Cluster nodes on multiple cluster networks</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
a:link, span.x_MsoHyperlink
{color:blue;
text-decoration:underline}
span.x_EmailStyle19
{font-family:"Calibri",sans-serif;
color:windowtext}
.x_MsoChpDefault
{font-size:10.0pt}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
div.x_WordSection1
{}
-->
</style>
<div lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="x_WordSection1">
<p class="x_MsoNormal">Thank you for the recommendation. Will try that out. Unfortunately the on-prem nodes cannot reach the head node via the public IP
</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">-Sajesh-</p>
<p class="x_MsoNormal"> </p>
<div>
<div style="border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="x_MsoNormal"><b>From:</b> slurm-users <slurm-users-bounces@lists.schedmd.com>
<b>On Behalf Of </b>Michael Gutteridge<br>
<b>Sent:</b> Friday, January 22, 2021 3:18 PM<br>
<b>To:</b> Slurm User Community List <slurm-users@lists.schedmd.com><br>
<b>Subject:</b> Re: [slurm-users] Cluster nodes on multiple cluster networks</p>
</div>
</div>
<p class="x_MsoNormal"> </p>
<div style="border:solid #9C6500 1.0pt; padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="x_MsoNormal" style="line-height:12.0pt; background:#FFEB9C"><b><span style="font-size:10.0pt; color:black">EXTERNAL SENDER</span></b><span style="font-size:10.0pt; color:black"></span></p>
</div>
<p class="x_MsoNormal"> </p>
<div>
<div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Courier New"">I don't believe the IP address is required- if you can configure a DNS/hosts entry differently for cloud nodes you can set:</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Courier New""> SlurmCtldhost = controllername</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Courier New"">Then have "controllername" resolve to the private IP for the controller for the on-prem cluster, the public IP for the nodes in the cloud. Theoretically anyway- I haven't run a config like that
and I'm not sure how the controller will react to such a configuration (i.e. getting slurm traffic on both interfaces).</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Courier New"">If the on-prem nodes can reach the public IP address of the controller it may be simpler to use only the public IP for the controller, but I don't know how your routing is set up.</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Courier New"">HTH</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Courier New""> - Michael</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-family:"Courier New""> </span></p>
</div>
</div>
<p class="x_MsoNormal"> </p>
<div>
<div>
<p class="x_MsoNormal">On Fri, Jan 22, 2021 at 11:26 AM Sajesh Singh <<a href="mailto:ssingh@amnh.org">ssingh@amnh.org</a>> wrote:</p>
</div>
<blockquote style="border:none; border-left:solid #CCCCCC 1.0pt; padding:0in 0in 0in 6.0pt; margin-left:4.8pt; margin-right:0in">
<div>
<div>
<p class="x_MsoNormal" style="">How would I deal with the address of the head node defined in the slurm.conf as I have it defined as
</p>
<p class="x_MsoNormal" style=""> </p>
<p class="x_MsoNormal" style="">SlurmctldHost=private-hostname(private.ip.addr)</p>
<p class="x_MsoNormal" style=""> </p>
<p class="x_MsoNormal" style="">The private.ip.addr address is not reachable from the cloud nodes</p>
<p class="x_MsoNormal" style=""> </p>
<p class="x_MsoNormal" style="">-Sajesh- </p>
<p class="x_MsoNormal" style=""> </p>
<div>
<div style="border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="x_MsoNormal" style=""><b>From:</b> slurm-users <<a href="mailto:slurm-users-bounces@lists.schedmd.com" target="_blank">slurm-users-bounces@lists.schedmd.com</a>>
<b>On Behalf Of </b>Brian Andrus<br>
<b>Sent:</b> Friday, January 22, 2021 1:45 PM<br>
<b>To:</b> <a href="mailto:slurm-users@lists.schedmd.com" target="_blank">slurm-users@lists.schedmd.com</a><br>
<b>Subject:</b> Re: [slurm-users] Cluster nodes on multiple cluster networks</p>
</div>
</div>
<p class="x_MsoNormal" style=""> </p>
<div style="border:solid #9C6500 1.0pt; padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="x_MsoNormal" style="line-height:12.0pt; background:#FFEB9C"><b><span style="font-size:10.0pt; color:black">EXTERNAL SENDER</span></b></p>
</div>
<p class="x_MsoNormal" style=""> </p>
<div>
<p>You would need to have a direct connect/vpn so the cloud nodes can connect to your head node.</p>
<p>Brian Andrus</p>
<div>
<p class="x_MsoNormal" style="">On 1/22/2021 10:37 AM, Sajesh Singh wrote:</p>
</div>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal" style="">We are looking at rolling out cloud bursting to our on-prem Slurm cluster and I am wondering how to deal with the slurm.conf variable SlurmctldHost. It is currently configured with the private cluster network address that the
on-prem nodes use to contact it. The nodes in the cloud would contact the head node via its public IP address. How can I configure Slurm so that both IPs are recognized as the head node?</p>
<p class="x_MsoNormal" style=""> </p>
<p class="x_MsoNormal" style=""> </p>
<p class="x_MsoNormal" style="">-Sajesh-</p>
<p class="x_MsoNormal" style=""> </p>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</body>
</html>