[slurm-users] CUDA environment variable not being set

[Sajesh Singh]

Relu, 
  Thank you. Looks like the fix is indeed the missing file /etc/slurm/cgroup_allowed_devices_file.conf



-SS-

-----Original Message-----
From: slurm-users <slurm-users-bounces at lists.schedmd.com> On Behalf Of Christopher Samuel
Sent: Thursday, October 8, 2020 6:10 PM
To: slurm-users at lists.schedmd.com
Subject: Re: [slurm-users] CUDA environment variable not being set

EXTERNAL SENDER


Hi Sajesh,

On 10/8/20 11:57 am, Sajesh Singh wrote:

> debug:  common_gres_set_env: unable to set env vars, no device files 
> configured

I suspect the clue is here - what does your gres.conf look like?
Does it list the devices in /dev for the GPUs?

All the best,
Chris
--
   Chris Samuel  :  https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.csamuel.org%2F&data=01%7C01%7Cssingh%40amnh.org%7C1bf5374fd6454b3fcd5a08d86bd6f427%7Cbe0003e8c6b9496883aeb34586974b76%7C0&sdata=INvZvw%2FiTrdf52patYRF9TtrQ0vuXRSivrxC8MJYLM4%3D&reserved=0  :  Berkeley, CA, USA