[slurm-users] Enforce gpu usage limits (with GRES?)

[Markus Kötter]

Hi,


limits ain't easy.

> https://support.ceci-hpc.be/doc/_contents/SubmittingJobs/SlurmLimits.html#precedence


I think there is multiple options, starting with not having GPU 
resources in the CPU partition.

Or creating qos the partition and have
MaxTRES=gres/gpu:A100=0,gres/gpu:K80=0,gres/gpu=0
attaching it to the CPU partition.

And the configuration will require some values as well,

# slurm.conf
AccountingStorageEnforce=associations,limits,qos,safe
AccountingStorageTRES=gres/gpu,gres/gpu:A100,gres/gpu:K80

# cgroups.conf
ConstrainDevices=yes

most likely some others I miss.


MfG
-- 
Markus Kötter, +49 681 870832434
30159 Hannover, Lange Laube 6
Helmholtz Center for Information Security
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5968 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20230203/ceb33912/attachment-0001.bin>