[slurm-users] Selecting only a subset of GPU's from all available GPU's

[Loris Bennett]

Hi Minu,

Minulakshmi S <minulakshmi.s at gmail.com> writes:

> I'm submitting jobs to a cluster via SLURM scheduler, and let's say I have access to 8 GPUs in my cluster in same node. They are GPUs of type A,B,C,D,E,F,G,H. I
> would like to submit a job that requests the use of GPUs of type A or B or C but NOT of type D/E/F/G/H. So I need some type of OR logic with the --gres flag.
>
> Eg : When I request GPU of type A , I can do sbatch –gres=gpu:TypeA:1,  I need to input a subset of GPU’s and let slurm schedule job utilizing one of the GPU
> from this allowed list.
>
> Regards
> Minu

Assuming the GPUs within a node are all of the same type, could you define
a feature for each GPU type, assign the features to the appropriate
nodes, and then run the job with 

   --contraint=gpu_TypeA  --gres=gpu:1

?

This is obviously rather clunky and it would be much nicer if
multiple GPUs types passed to '--gres' were ORed.

Cheers,

Loris

-- 
Dr. Loris Bennett (Herr/Mr)
ZEDAT, Freie Universität Berlin