[slurm-users] SLURM starts new job before CG finishes

[Erwin, James]

Hello,

I've recently updated a cluster to SLURM 19.05.4 and notice that new jobs are starting on nodes still in the CG state. In an epilog I am running node health checks that last about 2-3 minutes. In the previous version (ancient 15.08), jobs would not start running on these nodes until the epilog was complete and the node is out of the CG state. Does anyone know why this overlap of R with CG might be happening?

There is a release note for version 19.05.3 that looks possibly related but I'm not exactly sure what it means:

* Changes in Slurm 19.05.3
==========================
...
-- Nodes in COMPLETING state treated as being currently available for job
    will-run test.


Thanks,
James

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20200103/733c0db3/attachment.htm>