<div dir="ltr">I'm trying to spoof a gpu on a Centos 7.7 virtual machine that is a slurm node.  I just want slurm to see that this node has a gpu.  I'm not going to execute any code that uses a gpu.<br><br>I created a character device with:<br>mknod nvidia0 c 1 1<br><br>Here's what it looks like:<br>[root@liqidos-dean-node1 dev]# ls -l nvidia0<br>crw-------. 1 root root 1, 1 Jan 22 15:43 nvidia0<br><br>Here's my gres.conf:<br>Name=gpu Type=gp100  File=/dev/nvidia0 Cores=0,1<br><br>The relevant lines from my slurm.conf are (my full slurm.conf is below):<br>...<br>GresTypes=gpu<br>...<br>SelectType=select/cons_tres<br>....<br>NodeName=liqidos-dean-node1 Gres=gpu:gp100:1 CPUs=2 RealMemory=3770 Sockets=2 CoresPerSocket=1 ThreadsPerCore=1 State=UNKNOWN<br><br>After restarting slurmd it slurm doesn't recognize my spoofed gpu:<br><br>[liqid@liqidos-dean-node1 ~]$ slurmd -C<br>NodeName=liqidos-dean-node1 CPUs=2 Boards=1 SocketsPerBoard=2 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=3770<br>UpTime=0-06:47:11<br><br>[liqid@liqidos-dean-node1 ~]$ scontrol show node<br>NodeName=liqidos-dean-node1 Arch=x86_64 CoresPerSocket=1 <br>   CPUAlloc=0 CPUTot=2 CPULoad=0.01<br>   AvailableFeatures=(null)<br>   ActiveFeatures=(null)<br>   Gres=(null)<br>   NodeAddr=liqidos-dean-node1 NodeHostName=liqidos-dean-node1 Version=19.05.4<br>   OS=Linux 3.10.0-1062.el7.x86_64 #1 SMP Wed Aug 7 18:08:02 UTC 2019 <br>   RealMemory=3770 AllocMem=0 FreeMem=177 Sockets=2 Boards=1<br>   State=IDLE ThreadsPerCore=1 TmpDisk=0 Weight=1 Owner=N/A MCS_label=N/A<br>   Partitions=debug <br>   BootTime=2020-01-22T09:12:57 SlurmdStartTime=2020-01-22T15:55:16<br>   CfgTRES=cpu=2,mem=3770M,billing=2<br>   AllocTRES=<br>   CapWatts=n/a<br>   CurrentWatts=0 AveWatts=0<br>   ExtSensorsJoules=n/s ExtSensorsWatts=0 ExtSensorsTemp=n/s<br><br><br>Have I missed something, or is slurm smart enough to recognize that I don't have a real GPU?<div><br></div><div>Thanks.<br><br><br><br>Full slurm.conf:<br>SlurmctldHost=slurmctld-dean<br>GresTypes=gpu<br>MpiDefault=none<br>PluginDir=/usr/local/lib/slurm<br>ProctrackType=proctrack/cgroup<br>ReturnToService=1<br>SlurmctldPidFile=/var/run/slurmctld.pid<br>SlurmctldPort=6817<br>SlurmdPidFile=/var/run/slurmd.pid<br>SlurmdPort=6818<br>SlurmdSpoolDir=/var/spool/slurmd<br>SlurmUser=slurm<br>StateSaveLocation=/var/spool/slurm/state<br>SwitchType=switch/none<br>TaskPlugin=task/affinity<br>TaskPluginParam=Sched<br>InactiveLimit=0<br>KillWait=30<br>MinJobAge=300<br>SlurmctldTimeout=120<br>SlurmdTimeout=300<br>Waittime=0<br>SchedulerType=sched/backfill<br>SelectType=select/cons_tres<br>SelectTypeParameters=CR_Core<br>AccountingStorageType=accounting_storage/slurmdbd<br>AccountingStoreJobComment=YES<br>ClusterName=cluster<br>JobCompType=jobcomp/none<br>JobAcctGatherFrequency=30<br>JobAcctGatherType=jobacct_gather/none<br>SlurmctldDebug=info<br>SlurmctldLogFile=/var/log/slurm/slurmctld.log<br>SlurmdDebug=info<br>SlurmdLogFile=/var/log/slurm/slurmd.log<br>NodeName=liqidos-dean-node1 Gres=gpu:gp100:1 CPUs=2 RealMemory=3770 Sockets=2 CoresPerSocket=1 ThreadsPerCore=1 State=UNKNOWN<br>PartitionName=debug Nodes=liqidos-dean-node1 Default=YES MaxTime=INFINITE State=UP<br><div><br></div></div></div>