[slurm-users] SLURM: reconfig

[Tina Friedrich]

Hi List,

out of curiosity - I would assume that if running configless, one 
doesn't manually need to restart slurmd on the nodes if the config changes?

Hi Steven,

I have no idea if you want to do it every couple of minutes and what the 
implications are of that (although I've certainly manage to restart them 
every 5 minutes by accident with no real problems caused), but - 
generally, restarting the daemons (slurmctld, slurmd) is a non-issue, as 
it's a safe operation. There's no risk to running jobs or anything. I 
have the config management restart them if any files change. It also 
doesn't seem to matter if the restarts of the controller & the node 
daemons are splayed a bit (i.e. don't happen at the same time), or what 
order they happen in.

Tina

On 05/05/2022 13:17, Steven Varga wrote:
> Thank you for the quick reply! I know I am pushing my luck here: is it 
> possible to modify slurm: src/common/[read_conf.c, node_conf.c]  
> src/slurmctld/[read_config.c, ...] such that the state can be maintained 
> dynamically? -- or cheaper to write a job manager with less features but 
> supporting dynamic nodes from ground up?
> best wishes: steve
> 
> On Thu, May 5, 2022 at 12:29 AM Christopher Samuel <chris at csamuel.org 
> <mailto:chris at csamuel.org>> wrote:
> 
>     On 5/4/22 7:26 pm, Steven Varga wrote:
> 
>      > I am wondering what is the best way to update node changes, such as
>      > addition and removal of nodes to SLURM. The excerpts below suggest a
>      > full restart, can someone confirm this?
> 
>     You are correct, you need to restart slurmctld and slurmd daemons at
>     present.  See https://slurm.schedmd.com/faq.html#add_nodes
>     <https://slurm.schedmd.com/faq.html#add_nodes>
> 
>     All the best,
>     Chris
>     -- 
>     Chris Samuel  : http://www.csamuel.org/ <http://www.csamuel.org/> 
>     :  Berkeley, CA, USA
> 

-- 
Tina Friedrich, Advanced Research Computing Snr HPC Systems Administrator

Research Computing and Support Services
IT Services, University of Oxford
http://www.arc.ox.ac.uk http://www.it.ox.ac.uk