[slurm-users] Flag OVERLAP in advanced reservation

Gizo Nanava nanava at luis.uni-hannover.de
Thu Jul 27 19:50:36 UTC 2023


I observe strange behavior of advanced reservations having OVERLAP in their flag's list.

If I create two advanced reservations on different set of nodes and a particular username
is configured to only have an access to one with the flag OVERLAP,  then the username can
also run jobs on nodes in other reservation, which is reserved for other users.

In this example below, the username user1 should not have rights to start jobs on enos-n014:
> scontrol show reservations phd
ReservationName=phd StartTime=2023-07-23T08:00:00 EndTime=2023-07-23T20:00:00 Duration=12:00:00
   Nodes=phd-n[001-032] NodeCnt=32 CoreCnt=2048 Features=(null) PartitionName=phd Flags=FLEX,OVERLAP,SPEC_NODES,PART_NODES,MAGNETIC                                                                                             
   Users=user1 Groups=(null) Accounts=(null) Licenses=(null) State=ACTIVE BurstBuffer=(null) Watts=n/a

> scontrol show reservations test
ReservationName=phd StartTime=2023-07-23T08:00:00 EndTime=2023-07-23T20:00:00 Duration=12:00:00
   Nodes=enos-n014 NodeCnt=1 CoreCnt=32 Features=(null) PartitionName=(null) Flags=FLEX,SPEC_NODES,MAGNETIC
   Users=user2 Groups=(null) Accounts=(null) Licenses=(null) State=ACTIVE BurstBuffer=(null) Watts=n/a

user1 at login01:~$ salloc --nodelist=enos-n014
salloc: Pending job allocation 1816915
salloc: Nodes enos-n014 are ready for job
user1 at enos-n014:~$

It is interesting that once the job starts on the "wrong" node, enos-n014 in the above example, it has the field Reservation equal to phd.

> scontrol show job 1816915 | grep -i reser

If I remove OVERLAP (or user1) from the phd reservation, then user1 cannot run jobs on enos-n014.

We run SLURM 22.05.9

Any suggestion?. I would appreciate any help.

Thank you.

More information about the slurm-users mailing list