[slurm-users] [External] Re: Cluster nodes on multiple cluster networks

[Florian Zillner]

Chiming in on Michael's suggestion.

You can specify the same hostname in the slurm.conf but for the on-premise nodes you either set the DNS or the /etc/hosts entry to the local (=private) IP address.
For the cloud nodes you set DNS or the hosts entry to the publicly reachable IP.

example /etc/hosts on-prem node:
10.10.10.10   myslurmcontroller

example /etc/hosts cloud node:
50.50.50.50   myslurmcontroller

example slurm.conf for both locations
ControlMachine=myslurmcontroller
________________________________
From: slurm-users <slurm-users-bounces at lists.schedmd.com> on behalf of Sajesh Singh <ssingh at amnh.org>
Sent: Friday, 22 January 2021 22:17
To: Slurm User Community List <slurm-users at lists.schedmd.com>
Subject: [External] Re: [slurm-users] Cluster nodes on multiple cluster networks


Thank you for the recommendation. Will try that out. Unfortunately the on-prem nodes cannot reach the head node via the public IP



-Sajesh-



From: slurm-users <slurm-users-bounces at lists.schedmd.com> On Behalf Of Michael Gutteridge
Sent: Friday, January 22, 2021 3:18 PM
To: Slurm User Community List <slurm-users at lists.schedmd.com>
Subject: Re: [slurm-users] Cluster nodes on multiple cluster networks



EXTERNAL SENDER



I don't believe the IP address is required- if you can configure a DNS/hosts entry differently for cloud nodes you can set:



   SlurmCtldhost = controllername



Then have "controllername" resolve to the private IP for the controller for the on-prem cluster, the public IP for the nodes in the cloud.  Theoretically anyway- I haven't run a config like that and I'm not sure how the controller will react to such a configuration (i.e. getting slurm traffic on both interfaces).



If the on-prem nodes can reach the public IP address of the controller it may be simpler to use only the public IP for the controller, but I don't know how your routing is set up.



HTH



 - Michael







On Fri, Jan 22, 2021 at 11:26 AM Sajesh Singh <ssingh at amnh.org<mailto:ssingh at amnh.org>> wrote:

How would I deal with the address of the head node defined in the slurm.conf as I have it defined as



SlurmctldHost=private-hostname(private.ip.addr)



The private.ip.addr address is not reachable from the cloud nodes



-Sajesh-



From: slurm-users <slurm-users-bounces at lists.schedmd.com<mailto:slurm-users-bounces at lists.schedmd.com>> On Behalf Of Brian Andrus
Sent: Friday, January 22, 2021 1:45 PM
To: slurm-users at lists.schedmd.com<mailto:slurm-users at lists.schedmd.com>
Subject: Re: [slurm-users] Cluster nodes on multiple cluster networks



EXTERNAL SENDER



You would need to have a direct connect/vpn so the cloud nodes can connect to your head node.

Brian Andrus

On 1/22/2021 10:37 AM, Sajesh Singh wrote:

We are looking at rolling out cloud bursting to our on-prem Slurm cluster and I am wondering how to deal with the slurm.conf variable SlurmctldHost. It is currently configured with the private cluster network address that the on-prem nodes use to contact it. The nodes in the cloud would contact the head node via its public IP address. How can I configure Slurm so that both IPs are recognized as the head node?





-Sajesh-


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20210123/0bc8a63f/attachment.htm>