What version of Slurm are you running and what's the contents of your gres.conf file? 

Sean


From: lyz--- via slurm-users <slurm-users@lists.schedmd.com>
Sent: Tuesday, April 15, 2025 11:16:40 PM
To: slurm-users@lists.schedmd.com <slurm-users@lists.schedmd.com>
Subject: [slurm-users] Re: [EXT] Re: Issue with Enforcing GPU Usage Limits in Slurm
 
External email: Please exercise caution

Hi, Sean.
I followed your instructions and added ConstrainDevices=yes to the /etc/slurm/cgroup.conf file on the server node, and then restarted the relevant services on both the server and the client.
However, I still can't enforce the restriction in the Python program.

It seems like the restriction applies to the physical GPU hardware, but it doesn't take effect for CUDA.

--
slurm-users mailing list -- slurm-users@lists.schedmd.com
To unsubscribe send an email to slurm-users-leave@lists.schedmd.com