[slurm-users] "Low RealMem" after upgrade

[Ole Holm Nielsen]

On 10/5/21 8:05 AM, Diego Zuccato wrote:
> I already tried multiple times, both RESUME and IDLE, and it didn't work: 
> it just returned to "IDLE+DRAIN" with 'Reason="low realmem"'. :(
> I just tried again (after an unplanned shutdown of the frontend) and it 

What is a "frontend"?  Do you mean the slurmctld server?

> worked with IDLE (RESUME gives "Invalid node state specified").

So "scontrol update node=... state=idle" gives the node a correct idle 
state, whereas "state=resume" doesn't?  Did you restart the slurmd on the 
compute nodes?

> SLURM 20.11.4.

You wrote that you use Slurm 21.08 from Debian 11.  How did 20.11 get into 
the picture?  The slurmdbd and slurmctld servers must have versions >= 
that of slurmd, see some links in
https://wiki.fysik.dtu.dk/niflheim/Slurm_installation#upgrading-slurm

> Il 01/10/2021 21:32, Paul Brunk ha scritto:
>> If you mean "why are the nodes still Drained, now that I fixed the
>> slurm.conf and restarted (never mind whether the RealMem parameter is
>> correct)?", try 'scontrol update nodename=str957-bl0-0[1-2] State=RESUME'.

/Ole