[slurm-users] [EXT] GPU Jobs with Slurm

[Ole Holm Nielsen]

Hi Sean,

On 1/14/21 9:19 AM, Sean Crosby wrote:
> Hi Abhiram,
> 
> You need to configure cgroup.conf to constrain the devices a job has 
> access to. See https://slurm.schedmd.com/cgroup.conf.html 
> <https://slurm.schedmd.com/cgroup.conf.html>
> 
> My cgroup.conf is
> 
> CgroupAutomount=yes
> AllowedDevicesFile="/usr/local/slurm/etc/cgroup_allowed_devices_file.conf"
> 
> ConstrainCores=yes
> ConstrainRAMSpace=yes
> ConstrainSwapSpace=yes
> ConstrainDevices=yes
> 
> TaskAffinity=no
> 
> CgroupMountpoint=/sys/fs/cgroup
> 
> The ConstrainDevices=yes is the key to stopping jobs from having access to 
> GPUs they didn't request.

I'm just curious about your AllowedDevicesFile parameter, which doesn't 
seem to exist in the current Slurm versions 20.*.  Can you confirm that 
AllowedDevicesFile refers to an older Slurm version?

The gres.conf file handles device files currently, see 
https://slurm.schedmd.com/gres.conf.html

Thanks,
Ole