[slurm-users] CUDA environment variable not being set

[Brian Andrus]

do you have your gres.conf on the nodes also?

Brian Andrus

On 10/8/2020 11:57 AM, Sajesh Singh wrote:
>
> Slurm 18.08
>
> CentOS 7.7.1908
>
> I have 2 M500 GPUs in a compute node which is defined in the 
> slurm.conf and gres.conf of the cluster, but if I launch a job 
> requesting GPUs the environment variable CUDA_VISIBLE_DEVICES Is never 
> set and I see the following messages in the slurmd.log file:
>
> debug:  common_gres_set_env: unable to set env vars, no device files 
> configured
>
> Has anyone encountered this before?
>
> Thank you,
>
> SS
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20201008/8de61256/attachment.htm>