[slurm-users] Different GPU types on the same server

[David Gauchard]

Hello, FWIW we did this with gres.conf and slurm.conf:

in node's /etc/slurm/gres.conf:

AutoDetect=off
Name=gpu Type=quadro_k620 File=/dev/nvidia0 CPUs=0-0
Name=gpu Type=nvs_510 File=/dev/nvidia1 CPUs=1-1
Name=gpu Type=nvs_510 File=/dev/nvidia2 CPUs=2-2

in server's slurm.conf:

NodeName=gputesthost CPUs=4 RealMemory=3500 Sockets=1 CoresPerSocket=4 
ThreadsPerCore=1 Gres=gpu:quadro_k620:1,gpu:nvs_510:2

When submitting jobs, gpu is selected by user by that kind of request:
sbatch --gres=gpu:quadro_k620 ...

This is on a test host with old cards.
I don't know whether `CPUs` on each line should partition the host or 
not (=> CPUs=0-3 for all lines)

david

On 5/14/21 12:28 PM, Emyr James wrote:
> Dear all,
> 
> 
> We currently have a single gpu capable server with 10x RTX2080Ti in it. 
> One of our research groups wants to replace one of these cards with an 
> RTX3090 but only if we can give them a higher priority on that 
> particular card.
> 
> Is it possible to set up a queue that only includes a specific subset of 
> the cards on a server? Any way to use gres's etc. to achieve this?
> 
> 
> Many thanks,
> 
> 
> Emyr James
> Head of IT
> CRG - Centre for Genomic Regulation
> C/ Dr. Aiguader, 88
> Edif. PRBB
> 08003 Barcelona, Spain
> Phone Ext: #1098
>