[slurm-users] Enforcing GPU-CPU ratios

[Kilian Cavalotti]

On Tue, Jun 23, 2020 at 7:37 AM Bas van der Vlies
<bas.vandervlies at surfsara.nl> wrote:
>
> Which version of slurm do you use? as slurm 19.05:
>   * DefCpuPerGPU

Sorry for necroposting and undigging this old thread, but the
DefCpuPerGpu configuration option is actually just a default, which
will happily get overridden by job submission options. It's actually
reported as "JobDefaults" in `scontrol show partition`:

```
$ scontrol show partition foo | grep DefCpuPerGPU
   JobDefaults=DefCpuPerGPU=1
```

It works as a default:
```
$ salloc -p foo -G 3

$ echo $SLURM_GPUS_ON_NODE
3
$ echo $SLURM_CPUS_ON_NODE
3

```

but doesn't enforce the ratio:
```
$ salloc -p foo -G 2 -c 4

$ echo $SLURM_GPUS_ON_NODE
2
$ echo $SLURM_CPUS_ON_NODE
4
```

There is currently (as of 23.02) no mechanism to enforce a fixed GPU
per CPU ratio.

We've recently submitted a bug to request this feature (as a
MaxCpuPerGPU option, for instance), but we unfortunately won't be able
to sponsor its development.
If anyone's interested, it's up for grabs at
https://bugs.schedmd.com/show_bug.cgi?id=16189

Cheers,
-- 
Kilian