Hi Loris,

I have different kind of GPU's in same node , and I beleive feature applies to particular node , and cant be applied to only a few GPU's connected to a single node.

On Mon, 18 Dec 2023, 12:14 Loris Bennett, <loris.bennett@fu-berlin.de> wrote:
Hi Minu,

Minulakshmi S <minulakshmi.s@gmail.com> writes:

> I'm submitting jobs to a cluster via SLURM scheduler, and let's say I have access to 8 GPUs in my cluster in same node. They are GPUs of type A,B,C,D,E,F,G,H. I
> would like to submit a job that requests the use of GPUs of type A or B or C but NOT of type D/E/F/G/H. So I need some type of OR logic with the --gres flag.
>
> Eg : When I request GPU of type A , I can do sbatch –gres=gpu:TypeA:1,  I need to input a subset of GPU’s and let slurm schedule job utilizing one of the GPU
> from this allowed list.
>
> Regards
> Minu

Assuming the GPUs within a node are all of the same type, could you define
a feature for each GPU type, assign the features to the appropriate
nodes, and then run the job with

   --contraint=gpu_TypeA  --gres=gpu:1

?

This is obviously rather clunky and it would be much nicer if
multiple GPUs types passed to '--gres' were ORed.

Cheers,

Loris

--
Dr. Loris Bennett (Herr/Mr)
ZEDAT, Freie Universität Berlin