[slurm-users] [External] Re: Granular or dynamic control of partitions?

[Pacey, Mike]

Hi Feng,

Thanks - that's what I was looking for, though for my version of SLURM (23.02.0) it looks like the syntax is "scontrol update partition=mypart". Good to know that SLURM can cope with on-the-fly changes without affecting jobs.

With the "live" config now being different from the static I guess best practice is to ensure slurm.conf's partition definitions also need to be edited?

Regards,
Mike

-----Original Message-----
From: slurm-users <slurm-users-bounces at lists.schedmd.com> On Behalf Of Feng Zhang
Sent: Friday, August 4, 2023 7:36 PM
To: Slurm User Community List <slurm-users at lists.schedmd.com>
Subject: [External] Re: [slurm-users] Granular or dynamic control of partitions?

This email originated outside the University. Check before clicking links or attachments.

You can try command as:

scontrol update partition mypart  Nodes=node[1-90],ab,ac  #exclude the one you want to remove

"Changing the Nodes in a partition has no effect upon jobs that have already begun execution."


Best,

Feng

On Fri, Aug 4, 2023 at 10:47 AM Pacey, Mike <m.pacey at lancaster.ac.uk> wrote:
>
> Hi folks,
>
>
>
> We’re currently moving our cluster from Grid Engine to SLURM, and I’m having trouble finding the best way to perform a specific bit of partition maintenance. I’m not sure if I’m simply missing something in the manual or if I need to be thinking in a more SLURM-centric way. My basic question: is it possible to ‘disable’ specific partition/node combinations rather than whole nodes or whole partitions? Here’s an example of the sort of thing I’m looking to do:
>
>
>
> I have node ‘node1’ with two partitions ‘x’ and ‘y’. I’d like to remove partition ‘y’, but there are currently user jobs in that partition on that node. With Grid Engine, I could disable specific queue instances (ie, I could just run “qmod -d y at node1’ to disable queue/partition y on node1 and wait for the jobs to complete and then remove the partition. That would be the least disruptive option because:
>
> Queue/partition ‘y’ on other nodes would be unaffected User jobs for 
> queue/partition ‘x’ would still be able to launch on node1 the whole 
> time
>
>
>
> I can’t seem to find a functional equivalent of this in SLURM:
>
> I can set the whole node to Drain
> I can set the whole partition to Inactive
>
>
>
> Is there some way to ‘disable’ partition y just on node1?
>
>
>
> Regards,
>
> Mike