[slurm-users] [EXT] User association with partition and Qos

[Sean Crosby]

Hi Amjad,

Make sure you have qos in the config entry AccountingStorageEnforce

e.g.

AccountingStorageEnforce=associations,limits,qos,safe

Sean

________________________________
From: slurm-users <slurm-users-bounces at lists.schedmd.com> on behalf of Amjad Syed <amjadcsu at gmail.com>
Sent: Friday, 27 August 2021 20:28
To: slurm-users at schedmd.com <slurm-users at schedmd.com>
Subject: [EXT] [slurm-users] User association with partition and Qos

External email: Please exercise caution

________________________________
Hello all

We are having an issue understanding user association and partition.

Currently we have a partition with 30 GPU cards .

We have defined a qos gpu-rtx that allows user to reserve 2 cards


sacctmgr show qos gpu-rtx format=MaxTRESPU%60

                                                   MaxTRESPU

       -----------------------------------------------------

                                           cpu=96,gres/gpu=2




We have defined a user test that is assoc with this qos


sacctmgr show assoc user=test format=user,qos


Qos

gpu-rtx



Now we define another qos  gpu-rtx-reserved  that allows gpu=8


sacctmgr show qos gpu-rtx-reserved format=MaxTRESPU%60

                                                   MaxTRESPU

       -----------------------------------------------------

                                           cpu=192,gres/gpu=8

User test is not associated with gpu-rtx-reserved qos. So he should not be able to use more then gpu=2 .
Both of these qos are now in slurm.conf for the partition


parrtitionName=gpu-rtx6000-2 State=UP Nodes=g[15-29] MaxNodes=9 MaxTime=168:00:00 DefMemPerCPU=3996 AllowQos=gpu-rtx,gpu-rtx-reserved



But we found out that even though user is not assoc with gpu-rtx-reserved if the user uses gpu-rtx-reserved  in his slurm script , he can reserve 8 gpu cards


So our question is , can the users assoc with one partition qos can use the other qos in the partition  even if they are not associated with it . or in other words , we can only define one partition qos and not more then one.?


Hope i was able to explain ?


Any advice if we want partition to use more then one qos with different limits and users associated with one qos should not use other qos ?


Majid



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20210827/92b7fc84/attachment-0001.htm>