[slurm-users] Scontrol update GRES stops working on Slurm 20.02.5

[Chen Yang]

Hi,

I recently upgraded Slurm from 19.05.5 to 20.02.5 and I noticed that I am
no longer able to use the scontrol update command on GRES unless the new
count is 0.

For example, if I have a GRES named "test" defined and in slurm.conf

> NodeName=node1 Gres=test:2


Initially, I have:

$ sinfo -n node1 -O gres -h

$ test:2


After scontrol update, count remains the same:

$ sudo scontrol update NodeName=node1 Gres=test:1

$ sinfo -n node1 -O gres -h

$ test:2


Only updating to 0 seems to be working

$ sudo scontrol update NodeName=node1 Gres=test:0

$ sinfo -n node1 -O gres -h

$


It does not show any error or output but nothing is updated.

Non-zero Gres update was working for me on Slurm 19.05.5 and I did not see
anything related in the release note.


I only saw that both 19.05.5
<https://slurm.schedmd.com/archive/slurm-19.05.5/scontrol.html> and 20.02.5
<https://slurm.schedmd.com/scontrol.html> scontrol documentation mentions
this:

> Modification of GRES count associated with specific files (e.g. GPUs) is
> not allowed other than to set their count on a node to zero.

However, in my slurm.conf, Gres "test" is not specified with a file.


Is this the expected behavior of Slurm 20.02.5? Is there any possible
workaround (maybe explicitly clear out the 'file' field)?


Any help or comments would be greatly appreciated!


Best,

Chen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20200929/043c1696/attachment.htm>